Developing an Expert System as a First Step for AI

We are frequently asked by clients to incorporate Machine Learning and Artificial Intelligence into the applications we collaboratively develop. While we appreciate the opportunity this challenge presents, there is undoubtedly a process to navigate. This process includes evaluating the value of integrating AI into a system and assessing the available data for such tasks. The article below details our perspective and learnings from this journey.

In order to start implementing Machine Learning as part of a system, some initial questions need to be answered before selecting the approach and model to go for:

1. What do we want to achieve with Machine Learning?
2. What are the features (aka inputs) our model must expect and what are the labels (aka outputs) our model will produce?
3. Do we have enough data, and more importantly, reliable data for the training phase of the model?
4. Will this model learn/change over time?

In most of the cases, the answers to those questions are:

1. The requirements are almost clear.
2. We have some of the inputs clear but not all of them.
3. No, We do not have enough data (or even no initial data at all).
4. Yes, it will change over time.

The key entry point to develop and deploy a successful model using machine-learning techniques is having initial data for the training phase. When this is not available we need to look for alternatives to provide an optimal experience for our end users. This is where we pivot to another type of Artificial Intelligence called Expert Systems.

An expert system is a computer program that is designed to solve complex problems and to provide decision-making ability like a human expert. It performs this by extracting knowledge from its knowledge base using the reasoning and inference rules according to the user queries.

This type of Artificial intelligence does not require an initial dataset for training but instead a knowledge base and a set of rules in the form of if-else statements.

Our experience working with different companies and start-ups that want to implement machine learning has been that, in most cases, expert systems have proven to be the perfect solution in order to start introducing Artificial Intelligence into their business. This is especially true in the early stages for a few reasons:

1. The system provides results based on facts from the very beginning.
2. It is not as expensive to develop and maintain when compared to machine learning algorithms.
3. The quality of the results will depend on the quality of the knowledge-base which can be easily changed.

Successful use case

One of our clients is a fintech start-up with the vision to help families manage their investments with clarity and insights. The first strategy to start providing meaningful insights was to implement machine learning. By studying the portfolio of each Family, we would be able to create alerts/messages in real-time, providing useful information to their users.

With this clear goal, we started brainstorming the content that we would like to provide in these alerts and to check the data that we could expect from different portfolios so that we could sketch the first approach for our system. After this process we developed the first version for generating such alerts using an event-driven architecture and AWS SageMakerWith this clear goal, we started brainstorming the content that we would like to provide in these alerts and to check the data that we could expect from different portfolios so that we could sketch the first approach for our system. After this process we developed the first version for generating such alerts using an event-driven architecture and AWS SageMaker for the machine learning model.

While training and testing our model, we found some challenges:

1. Banking data is not up to date.
2. Same data can be presented differently between banks.
3. We need tons of historical information about each portfolio in order to start providing useful insights.
4. Low quality of the data received from banks.
5. Lots of variables can impact a portfolio (politics, economics, speculation, user stocks preferences, etc.).
6. Outdated insights are useless.

.... and the list continued to grow.

We quickly realized that in order to provide smart insights to the Families based on the information available and the fact that the timing of when these insights get delivered is extremely important, we had to develop a system where we could code some known rules that could be triggered based on the information we receive about each portfolio. We ended up building an Expert System.

After almost four years of development, the result is a simple but highly effective, low cost system to generate near real-time insights that get delivered to our users based on the knowledge and experience acquired by the experts and later coded into the system. Some users have reported that they got useful information about their portfolios even BEFORE they got it from their actual advisors, proving that machine learning is not the only way to accomplish artificial intelligence that can meet user expectations and increase user engagement.

What’s next…?

We now have a better understanding of how our end-users are using the system, what the quality of the data is we are getting from banks, what the most common inconsistencies we receive from bank accounts are and more importantly how to address most of those errors. Still, the question remains: How can we continue delivering reliable information to our users if the source of information is not accurate? And also, how will we know when the next error/unexpected behavior pops up because of “new inconsistencies” coming from the banks that we do not know about? Are we going to keep adding more “workarounds” to overcome every single scenario where things go wrong? And, more importantly, is this approach scalable with 1000+ users connecting their bank accounts every day? This list of concerns will continue to grow.

Having an expert system is not enough in this case, as the knowledge base is finite and you need experts to continue monitoring the data, figure out a solution and either update the knowledge base OR implement a solution for it which can take days, weeks or even months of development effort. We have reached the point at which machine learning can truly add value to the business.

By evaluating the challenges we are facing and the experience and knowledge we have gathered over the years, we went back to the four questions we mentioned above, and this time we got a different set of answers:

1. We want to detect inconsistencies in the data we collect from bank accounts and alert our users of such inconsistencies so they can correct them by themselves or confirm that those are in fact expected behaviors.
2. The input is the current status of each portfolio (balance, cash, transactions, securities, …) and the output is whether there is an inconsistency and where, or not.
3. This time we have historical information about the user’s portfolio, and users have been making changes to the original data in order to get their portfolio to the desired state. So now we have data to train our model and, more importantly, know what would be the expected result for every user.
4. Yes, it will change over time based on the information we keep collecting and the feedback we collect from our users regarding the output of our model.

Having answered these four questions, the goal now is clear on where we want to apply machine learning, so the next step will be to define our stack of technologies to implement an algorithm to detect data anomalies. By using recommenders and data anomaly algorithms, our goal is to find outliers in the data that could indicate that there are some transactions that are wrong and more importantly, make suggestions on what the expected state should be.

Summary

In summary, this journey started with the end goal in mind and then worked backwards to get the best possible solution given the state of knowledge at each phase of the project. True Machine Learning and Artificial Intelligence will only be possible once the team understands the true issues, builds up enough data to drive the model, and figures out the corner cases. It is a long journey for sure, but absolutely justified by the results.