Changi General Hospital

Visionary Schoolmen

Objective

• To achieve a consistent and competent level of triage skills across the spectrum of nurses.
• By creating a suite of assessment module with blended learning, we hope to leverage on machine learning and artificial intelligence (A.I) to dissect and understand why and how triage decisions are made, and whether it is affected by the personal risk profile of the nurses.

Current Situation and Challenge

Changi General Hospital has a busy Emergency Department which sees an average of 450 patients per day. With such a high workload, triage is an important and essential process as it prioritizes patients according to the urgency and seriousness of their illness. This is done as a front line process by a triage nurse. Consultations with doctors only happen after the initial triage process.

For 90 % of the patients, triage is the first step of a patient’s journey in the ED. It consists of a brief assessment, including a set of vital signs, and the assessment of a "chief complaint" (e.g. chest pain, abdominal pain, difficulty breathing, etc.). After this step, patients are assigned a triage priority number which determines how urgently they should be attended to. Most emergency departments have a dedicated area and specially trained nurses to carry out this process.

Inaccurate triaging of patients results in either over or under triaging. The former implies that patients are labelled as more ill than they actually are and that they require more urgent care. This leads to a waste of resources and decreases efficiency of the department. More concerning is the latter: under triaging would result in delays in management of truly ill patients, leading to poor health outcomes and safety infringements.

Envisaged Prototype

Our aim is to achieve a consistent and competent level of triage skills across the spectrum of nurses. We propose an assessment module with teaching embedded in it, gamification to increase the motivation to participate and artificial intelligence at the backend to capture data on the decision making process, so as to recommend learning pitched to the learner’s milestone of maturity.

Through consistent and competent triage, we hope to bring value to the hospital, healthcare workers and most importantly to our patients in the following ways:

• Decrease Triage duration/time
• Improve triage accuracy
• Shorter waiting time to Doctor’s consult

Lien Centre for Palliative Care

Duke-NUS Medical School

Interactive Digital Centre Asia

School of Engineering

Temasek Polytechnic

Objective

As a premier palliative care research and education center, LCPC recognizes the importance of providing effective communication skills training to healthcare professionals. Evidence suggests that when healthcare professionals communicate effectively with patients living with life limiting illnesses, patients report (1) less decisional conflict; (2) higher quality of life; and (3) lower expenditures in healthcare spending.

Current communication skill training in serious illness conversations is generally technology light. It often relies on role play by paid actors and small group discussions with qualified facilitators. This approach, while effective, is labor intensive, and not scalable. It also does not allow multiple practices in a simulated environment because organizing such workshops with standardized patients (actors) is resource intensive.

Therefore, our objective is to develop an intelligent simulation model that can effectively teach communication skills training without the need for local actors and on-site professionals. This platform should incorporate the latest gaming and chat-bot technology to simulate authentic case scenarios in place of real actors in a format which mimic and gesture like real-life patients and healthcare professionals. This approach represents a novel strategy for training healthcare professionals in serious illness conversations in a scalable and resource efficient manner. If successful, it can easily be adapted to other patient groups locally and regionally.

Current Situation and Challenge

Effective communication with patients has been shown to improve quality of care, lower healthcare costs and enhance the patient experience. However, studies have shown that current structured communication skills training is often ineffective due to the lack of time and competing priorities. Studies have shown that communication skills are essential in end-of-life (EOL) care. In the current blended learning curriculum provided by Lien Centre of Palliative Care (LCPC) at Duke-NUS medical school, communication training is emphasized in all its courses. This training is carried out using a variety of teaching methods including case studies, online learning, role-play and discussion. Despite positive feedback on the quality of the training, there is a limit to the numbers of learners being taught at a time.

Envisaged Prototype

The proposed solution integrates proven emerging technologies into a platform that potentially will solve the healthcare communication problems highlighted above. Chatbot engine development has taken a big leap in recently years and are now deployed successfully in social, service and education applications. However, chatbots used are usually deployed on computer screens or mechanical robots. The proposed solution aims to integrate a chatbot engine with 3D technology, creating avatars that have human-like dynamic expressions and intelligence in natural language processing(NLP).

Together with speech recognition and advanced text to speech synthesizer, the avatar will be able to eventually replace the role-play actor. Hence, learners using a tablet computer, will be able to converse naturally with the intelligent avatar (playing the role of a patient with serious illness) and get an audio response.

An intelligent speech analysis module will put the learners’ speech into context for the avatar response as well as for assessment according to a predefined framework. This is expected to solve the problem of:

• scaling and cost savings (replace actors, savings on venues, trainers’ time)
• efficiency, allowing learning anywhere (tablet allows mobility) for LCPC.

Assessment of trainee will not only involve the conversation recorded during the training. The proposed AI assessment system will collect conversation data, trainee facial expressions (through the camera in the tablet) which will be fed through machine learning and NLP algorithm network. With this system established, it can be used to analyse training and learning gaps and improve itself for future training. In addition, the data collected will serve to highlight key areas of weakness in the training, the training profile of learners and will overall be a more systematic approach to teaching and learning in healthcare communications. Hence the following potential problems can be addressed for the trainers:

• productivity in teaching and learning activities (used to be singular assessor for multiple learners; now each learner can do it on their laptop or tablet increasing productivity)
• validity and reliability of assessment (due to assessor fatigue, different assessor, with a platform approach, the validity and reliability of scoring can be enhanced)
• data collected can identify gaps to fine tune teaching and learning

The platform could be expanded to different healthcare professionals and different areas in the healthcare industry to enhance healthcare communications between healthcare professionals and patients. The potential of the CHAT-SG platform could also be used in other industry verticals that have key needs in effective communications.

Games are increasingly used in training because of their ability to engage and immerse learners in the game world. Games are effective teaching and training devices for learners of all ages because they are highly motivating and engaging. Serious game is designed primarily for the purpose of education and training. The purpose is to encourage behaviour change rather than for pure entertainment. It is widely used in advertising, healthcare, military, non-profit organization as well as in education. It allows players to assume realistic roles, face problems, formulate strategies, make decisions and get feedback on the consequences of their actions.

ELDO Consulting

One-Third Technologies

Objective

• Develop portable, immersive and mobile learning solutions for food preparation and handling kitchen equipment
• To overcome barriers to learning for the disadvantaged and low-income citizens, hence improving their education and employability outcomes.
• To overcome limitations of high operational costs for training organisations conducting vocational skills, in a collaborative kitchen setting

Current Situation and Challenge

As a social enterprise, mSENSE reaches out to job seekers and equip them with relevant skills, before placing them in recruiting industries. Many job seekers lack proper qualifications and/or face difficulties in acquiring new skills in industries that has now deployed new machinery and digital technologies. Learning opportunities, especially for vocational skills in F&B, are limited due to high investment and time required to set up real kitchens (Eastern, Middle Eastern, Western kitchens). Even though mSENSE has partnered a training organisation with proper kitchens, hands-on learning opportunities are still limited, as schedules require them to come at different days, or are may be inconvenient to access. In some cases, the trainers may not be allowed to handle actual equipment at the facilities. These concerns are further accentuated for mature job seekers who may be slower learners and require more hands-on time.

Envisaged Prototype

The idea is to develop a set of procedural and collaborative simulations for a team of cooks preparing food and handling equipment (knife, plates, frying pan, etc) in a virtual but realistic kitchen environment.

As new smartphones are now built with capability for augmented reality, the project team proposed to build a new mobile learning management system that will feature integrations to XR applications and content library of media-rich assets for the food industry courses. The mobile LMS will include unique authentication protocols that continuously check on the identity of learners engaged in the course.

Current protocols such as biometrics and 2FA are unable to convince training organisations on the viability of self-taken eAssessments and self-directed learning activities. A mix of learning games (e.g. cooking processes) and rich media assets will be developed for a mobile-based micro-learning course. This will enable mSENSE to plan various blended learning strategies and short course offerings to various groups of learners.

SCAL Academy

ReVR

FutureSafe Tech

CognaLearn

Objective

Using innovative technologies to evaluate learning outcomes of safety training for higher safety performance

Current Situation and Challenge

• Tech-enabled immersive training is mainly used for training but not for evaluation of training
• Training evaluation by training provider remains largely at Level 1 (using end of course evaluation form) and Level 2 (using end of course assessment). Training provider is unable to measure behavioural change of learners (Level 3) in the classroom.
• Employers are unable to monitor and predict psychometric triggers of accident hence are unable to prevent accident from happening

Opportunities:

• Ability to measure psychometry of learners during training with the use of tech-enabled immersive learning environment and wearables IoT
• Ability to carry out data analysis using artificial intelligence (AI)
• Ability to use the data from AI to predict and hence prevent accident

Envisaged Outcomes:

• Ability to measure training effectiveness in behaviour modification in classroom setting (which is not possible without technology)
• Ability to use psychometry to monitor and predict accident and hence raise overall safety performance of the industry

Envisaged Prototype

• Combining available technologies (i.e. immersive, wearable IoT and AI) to interpret and evaluate learning outcomes which cannot be done using traditional way of learning assessment
• Using AI to predict and hence prevent accident

The learners will be immersed in hazardous work at height and confined space environment (using VR and simulated learning) while wearable IoT is used to measure and monitor the learners’ psychometry (heart rates). The output of the wearable is then fed into the AI software for data analysis

Shalom International Movers

Dioworks Learning

Objective

To map and develop competencies among staff to prepare for an Al-based workplace using technologies such as Chatbots for learning and Al-based visual recognition simulation systems.

Current Situation and Challenge

The current situation does not fully map out the needs of the new hires coming into the moving and logistics industry. This leads to issues concerning capability development and how automation can then fit with the strengths of the present and future staff. In short, there is little or no leverage of staff strengths and automation to improve productivity. The bigger question is what happens to current and new staff as the company prepares for automation. At the same time, with automation, customers need to be informed and educated about how the new process flow works. They will play a critical in ensuring the robotic process automation is successfully implemented.

Envisaged Prototype

Firstly, the innovation begins with mapping out the roles of intelligent machines and automation (e.g. carrying heavy loads and navigating paths autonomously; holding lift doors and wrapping up objects with care) within the 1-year, 3-year and 5-year timeframes.

Secondly, examining how humans work with intelligent machines to perform the roles pegged at higher levels (e.g. during contingencies or working with really costly items), there is a need to review the praxis between the two agents within the moving environment.

Thirdly, the training map to get human staff ready for the new roles needs to be designed and developed. For example, the concept of RQ— Robotic Quotient (in the vein of IQ and EQ) could be useful, to uncover how humans need to cope with robots and Al at work.

Fourthly, the training map can then be translated into learning activities and resources using different AI-based tools to drive learning.

The innovative tools for delivering the training should also be Al-based and immersive:

• Chatbots to train staff and customers
• Visual recognition for learning simulations
• Kinetic Coach to practise psychomotor skills with feedback.

These tools are responsive, adaptive and provide a higher level of learner engagement to build expertise rather than competencies. Going forward, competencies are likely to be replaced by automation while humans will take on more expert roles. Hence, the training approach should be expertise building. Examples of these pedagogies will have to be developed. One example could be error-based learning.

The approach used is innovative because it is future-oriented, with a careful mapping of future requirements of capability-building so that the tools used can be focused on developing these expertise (e.g. meta-cognition, pre-empting errors and being innovative).

Pedagogically, these tools and approaches are based on rich literature / research (e.g. Dreyfus' Novice to Expert Model, literature on Productive Errors

Singapore Insitute of Management

Objective

To develop 21st Century Competencies in both local and global learners within the context of a rapidly evolving jobs landscape using experiences and outcomes enacted via an AI-based approach.

Current Situation and Challenge

The current gaps in learning and teaching include:

• Time lag in helping learners acquire just-in-time competencies based on the latest technologies and know-how – the training is always a step behind what the industry needs, especially for cutting edge technology and competencies
• Both classroom and online learning tend to be didactic and this reduces the impact of the learning, for workplace application. In contrast, workplace learning provides a more experiential and application-based approach but the cost is extremely high and the outcomes inconsistent. Can AI-based learning tools bridge this gap and marry the strengths of all three approaches?
• The typical outcomes of most formal learning and teaching tend to be associated with certification, often backed by an educational institution. However, this obscures the true capability of the learner which are often encapsulated within work products and portfolios. Learners do not have the necessary skillsets (21st Century Competencies) to perform work in the contemporary workspace and environments. They also lack understanding of AI and new technologies such as blockchain.

The outcomes are multifold:

• New micro-learning pedagogy
• Integrated learning, experiential and outcome-based approach to acquiring 21st Century Competencies
• New credentialing system to recognise work / portfolio competencies beyond certificates

Envisaged Prototype

The innovation is to drive the development of 21st Century Competencies in local and global learners with the use of engagement structures helmed by AI-based technology in a scalable, sustainable and fun manner. The evidence of the competencies is tracked using chatbot technology and demonstrated when learners designed their own chatbots to showcase their newly acquired competencies. The eventual outcome is a credentialing system that documents the learners’ capability for global recognition.

The proposed solution involves 3 integrated levels of learner engagement:

• A spiral curriculum underpinned by 2 chatbots for learning to help learners understand and demonstrate the key concepts in 21st Century Competencies, which includes working with AI
• Regular conversations with learners via AI-based engagement structures to get learners to develop critical behaviours and subsequently habits of mind for long-term transformation
• Simple-to-use platform for learners to design their own chatbots for learning as part of their demonstration of 21st Century Competencies – with this, learners acquire hands-on experience with AI and AI-based communication competencies

Finally, the credentialing platform links up with the Credentialing Blockchain that is Social Media Friendly to heighten the motivation learners have to make the topics worth their learning.

The learner responses drawn from across all three levels of engagement are documented to be added to the database for credentialing of the learner subsequently. Online assessment by SIM Global Education lecturers allow their learners to be certified as having achieved the necessary competencies.

Singapore Institute of Power and Gas

CognaLearn

Objective

The overall objective of the project is to tighten and strengthen the feedback loop between training experience and on the job performance with three goals:

• For Learners: Enhance their transfer of learning by facilitating performance feedback after training back in the workplace
• For Trainers: Allow trainers to adjust their delivery of training based on feedback
• For Business Owners: Understand the transfer of learning performance to workplace performance

Current Situation and Challenge

Issue 1: Adjustment of delivery strategies for long courses

SIPG conducts several courses which span over a long duration (e.g. 20 days over 5 months) and involve one or more trainers. If a single trainer conducts the entire course, typically, the trainer will adjust the training materials and delivery methods based on his assessment of the initial sessions. If the training is conducted by different trainers for each segment, the trainers of the earlier sessions may share their insights with other trainers if they find the opportunity to do so.

Issue 2: Understanding the transfer of learning performance to workplace performance.

While SIPG collects data on learners' reactions and their learning at the end of its training programmes, it faces challenges in the assessment of the transfer of learning performance to workplace performance, and subsequently, impact on the business performance due to the need for time to lapse before the assessment can be made.

At the time point identified, the course evaluation team will email or call the learner and/or supervisor to find out about the changes in workplace performance of the learner. The staff will then collate the responses before assessing the effectiveness of the training the learner had attended. The results of this assessment is then fed back to the courseware and training team for their refinement of courseware and training delivery.

Issue 3: Provision of regular feedback on transfer of learning performance to workplace performance

Typically, once the course ends, the support for the learner to continue to apply the skills and knowledge gained in the course depends heavily on the support of the supervisor, who needs to provide opportunity for the learner to apply what he has learnt. However, even if the opportunity is provided, as in common in Asian societies, constructive feedback is provided more frequently than positive reinforcements. To obtain comprehensive feedback, learners need to proactively ask their peers and supervisors for feedback on their performance. As most Asians are shy about asking for feedback and providing positive feedback, this becomes a challenge for learners who require support and encouragement for their efforts in applying what they have learnt.

Envisaged Prototype

The innovation consists of three key features, namely

• Providing immediate insights for trainers to provide adaptive training in terms of both training content and training methodology (meeting Issue 1)
• Use of data analytics in the measurement of training effectiveness (up to Kirkpatrick Level 3) (meeting Issue 2) 
• Facilitating the transfer of learning by providing an avenue for feedback relating to the topic (meeting Issue 3)

Success indicators

Learners: Ability to answer the questions: "I went off for training, did I get better when I got back to my job?" and "Am I applying what I learned correctly?"

Trainers: Ability to answer these questions:

• Which areas were learners able to figure out on their own (with pre-work)?
• Which areas were learners able to figure out working in a team?
• Which areas are trouble spots that trainers should focus on (in real-time and delayed)?
• Which assessments or assessment items are most effective at predicting on-the-job performance?
• How effective was the training in on-the-job performance

Business owners: Ability to answer the question "I sent an employee for training, were there able to perform better on the job after the trainer or not?"