Although significant resources are spent today on development of AI-based automation, these efforts frequently fall short of the desired AI systems – ones which accurately implement the relevant human knowledge, experience and decision making process. Advancements in AI models creation are mainly focused on machine learning tools, neural network architectures used, and training data base; with the later one often being nonspecific, performed outside of the customer operation scope and not sufficient in size. However, if the goal is to reach AI models that could augment or replace the human user, addressing specific customer needs and criteria, then incorporating the human expertise is the key to success. Therefore, the ultimate challenge is to establish a direct interface between the expert user and the AI system, an interface that will enable a smooth and accurate User-to-AI knowledge transfer. Such an interface is valuable, not only for development and training of AI models, but whenever a combined AI-human decision is needed for a decision-making process, i.e. in cases where the AI system doesn’t reach the required accuracy or when liability, regulatory and similar restrictions require a human-in-the-loop approach.
Brain in the Loop AI System
With this mindset Israel-based InnerEye (distribution Macnica ATD Europe) developed a “brain in the loop” AI-powered system that enables customers’ experts to take an active role in the creation of AI models and in their operation. The company is convinced that the foundation for developing and validating any AI model is intimate online collaboration between humans and AI, specifically the users’ brain activity and AI. “Even in daily life, every time we talk to Siri, when we use smart home applications or provide input to a search engine, there is human interaction with AI to create a decision. InnerEye subsequently takes this to the next level by combining human intelligence with artificial intelligence in a self-learning system where both the human and the computer vision inputs are processed synchronously”, explains InnerEye CEO Uri Antman.
Improving AI Training and Inference
The system layout is parallel with an image data stream enriched by the decoded brain activity data measured through a wearable EEG headset used by the expert user during system operation. Both data streams are processed by the InnerEye Brain-AI engine which processes in parallel the image data and the associated user brainwaves. This dual approach ensures much higher efficiency and accuracy of both training and decision making, as benchmark projects have proven in the fields of security as well as in industrial, aerial imagery and medical applications. The feedback from the human brain activity continuously provides inputs used to improve the deep learning algorithms of the AI models. This provides the additional benefit of bringing the AI model out of the “black box” status. The additional information from the brain activity provides richer labelling with more information on the image. “This is particularly relevant wherever decisions go beyond pure binary yes-no judgement and the EEG brain information provide what we call a soft label or user confidence level to each data point”, says Uri Antman. Moreover, the fact that the system implements a loop between the human brain and the AI model, allows a continuous training of the AI system. And, last but not least, the training and update of the AI models is done individually by the customers’ experts using customers’ data upon the customers’ own application. This is the most unique difference to standard training methods of AI systems, as it personalizes the AI system by the data continuously streamed from the customer users brains.
Better than Classic AI Systems
Moreover, InnerEye’s unique brainwave analysis is able to detect the users’ levels of concentration and attention on task, thus providing a level of confidence on the user annotation and decision – mitigating risks of “bad lessons” to the AI system or inaccurate operation decisions. Altogether, this innovative approach, combining human and machine factors into one system, turns out to be more productive and less costly than classic AI systems. The combined system accuracy increases compared to classic AI systems and requires less data and training cycles.
