Sequence-Based Learning

Ada:

Hi, it’s Ada Lovelace here again! Since the ADA Lovelace Center for Analytics, Data and Applications is named after me, I’d like to find out more about the domains that are grouped together in this project and about the applications they can be applied to. Today, I’m chatting to Chris. He works at the ADA Lovelace Center and is an expert in several of the Center’s AI focus areas. I’m keen to talk to him about sequence-based learning. Chris, could you explain how the method works?

 

Chris:

Sequence-based learning focuses on the temporal and causal relationships that exist within data, particularly in applications such as language processing, event processing, biosequence analysis and multimedia data. It works on the basis of data series that can be extrapolated with respect to a further dimension. Observed events are used to determine the system’s current status and to predict, or extrapolate, future conditions. This is possible in cases where the sequence of the events is known, as well as in cases where they are provided with exact time stamps. That’s ultimately what makes this method so special. Unlike other machine learning methods that learn by taking random data points from a given data set, for example from a giant database, sequence-based learning relies on the order or sequence of data as an essential part of the process. So you can’t just randomly swap one piece of data from the database for another.

 

Ada:

Does that mean only certain types of data can be used with this method? Which types is it suitable for?

 

Chris:

From a purely technical point of view, I would say the method can be applied to any type of data that evolves with respect to some kind of dimension, though there will inevitably be some cases where its application doesn’t make sense. Consider how a video is really just an image that we extrapolate in another dimension, in this case time. This kind of approach has also been used on a more general basis to complete images, essentially building up the image line by line.

Those examples might sound somewhat theoretical, but there are actually lots of situations in which we have access to sequential data. The analysis of texts and sentences in language processing is one example. Others include recognizing phonemes in speech signal processing, evaluating inertial and magnetic-field sensor arrays in sensor signal processing, identifying the temporal causal dependency of events in positioning data, analyzing the evolution of commodity prices, working with genome sequences, and a huge number of other applications.

 

Ada:

I see. When you say dimensions, do you always mean time? Or can you also use other dimensions?

 

Chris:

Well, an image consists of rows and columns, for example. But it’s also possible to define the image as a sequence consisting of a series of pixels arranged one after the other. Doing that basically gives you a character string, and the interesting thing is that these strings can be learned and the model can then be used to make predictions. In concrete terms, that basically means you can take an incomplete image and complete it. Other areas where we can’t really talk about a temporal dimension are genome sequences and texts, because in those cases we’re normally concerned with the position of an individual genome or word within a sequence.

 

Ada:

Great! So, now we’ve talked about the types of data involved, but I’m still wondering what kind of quantity and quality of data you need.

 

Chris:

Obviously, that depends on the particular method in each case, but the answer basically runs along the same lines for all ML procedures; that is, the deeper and more complex the model, the more data you need. And the more complex the structure behind the data, the more complex a model you need to really capture the full complexity. For example, there are Gaussian processes you can use to capture temporally or spatially distributed data with nothing more than probabilities and uncertainties. This approach doesn’t really require much data, which often makes it a very attractive option. On the other hand, there are deep learning methods such as recurrent neural networks in all their many forms. Using these properly requires much, much more data and a tremendous amount of expertise. But if you get it right, you can achieve truly amazing things!

 

Ada:

So, the outcome of the model depends on many factors. But assuming that these factors are optimal, what can I achieve with this approach? What kind of result does it produce?

 

Chris:

There are many possible results depending on the method used. For example, we can use time-series classification to capture complex human motion, and forecasting to predict commodity prices. We can also analyze scenes in soccer games because the moves and behaviors in soccer can actually be described in very concrete ways. Here at the ADA Lovelace Center, we are also specifically concerned with developing ways to find similar scenes as quickly as possible or to suggest possible scenes that haven’t been seen before. Ultimately, this approach can provide genuinely useful assistance to trainers when they’re working with their team.

 

Ada:

Aside from your sports example, what other potential applications are there? Which areas or industries are these methods suitable for?

 

Chris:

These methods are actually applied relatively frequently. My perception is that data sequentiality – in other words, causal relationships between individual data points as opposed to some kind of gigantic data set – very often has a key role to play. We often have access to data points that are temporally related to each other and that therefore lend themselves to longitudinal and cross-sectional analyses. As well as sports analysis, you could also use this approach to delve deeper into biosignal analysis: in ECG analysis, for example, you have different events that occur in the signal and that are essentially causally related to each other. Other examples include production and machine data analysis; for example, if I model a job sequence using an inertial sensor system, then I can also detect temporal relationships between individual steps in the process and verify the attainment of a certain level of process quality.

 

Ada:

From what you’ve said, it seems that this area of expertise can be used in many different domains. Does it also combine well with other AI methods? If so, which ones? Is that something researchers at the ADA Lovelace Center are already starting to investigate?

 

Chris:

From a practical point of view, these methods can be combined with all other methods. For example, sequence models are also used in reinforcement learning within the broader context of experience-based learning. But we also have a lot of interaction with traditional fields of research such as few labels learning, because the challenges of time series often throw up their own very specific problems. My feeling is that we’re still very much in the early stages in the explainable learning domain. Time-series models certainly work really well, but those that work really well generally tend to be the ones that are also very, very difficult to interpret.

 

Ada:

Thanks so much for those fascinating insights! From what you’ve said, it seems to me that it’s not just a question of continuing to develop the methods and models and getting them into practical use, but also of combining them with other methods of artificial intelligence. Experts at the ADA Lovelace Center conduct research in a total of nine different domains, and you’ve agreed to help us find out more about another one of those domains: experience-based learning. I look forward to talking to you again!