What is 4D tomography and what benefits does it offer?
Prof. Tomas Sauer: 4D tomography is another way of saying “time-resolved 3D tomography.” In concrete terms, 4D tomography is about producing imaging not in the form of 2D images but rather of 3D objects per time frame. The fourth dimension is time, which can be over very long timescales or indeed very short ones, such as for the decomposition process of a foam. This project is about images produced at synchrotron facilities. We’re not talking about regular lab CT data but rather much higher resolutions and better quality all around. What makes this challenging is that the extremely high temporal and spatial resolution leads to massive amounts of data – terabytes or potentially even petabytes. In the case of the BM18 synchrotron, every second of imaging generates around 2 GB of data.
The technology is useful for observing anything that changes over time and precisely following these changes at the microscopic level. Using 4D tomography, it’s possible to track, evaluate, and analyze the changes right down to the micrometer range.
What’s the problem with the data gathered?
Prof. Tomas Sauer: Quite apart from the tremendous size, there are three main problems: image noise, artifacts, and the sheer complexity of computing the data. Everything is three-dimensional, which from a mathematical perspective means now everything grows in terms of the third rather than the second power – it’s cubed not squared. This doubles the resolution once again.
Many of the well-established methods for using AI in image analysis, segmentation, and so on work for 2D, but often not for 3D data. This is mainly because the number of parameters for 3D is significantly higher. If I want to use AI methods to look at volumes, I know that the complexity and the number of parameters will greatly increase as well. A knock-on effect is that more training data is required. That’s when I’m faced with the problem that the amount of training data I collect is larger and is more difficult and expensive to obtain. Datasets for a really good 4D CT are not exactly a dime a dozen.
How can AI help and what exactly are you working on in the KI4D4E project?
Prof. Tomas Sauer: Basically, artificial intelligence can help make the data more accessible. We apply a variety of methods to achieve this. One of the goals that we’re pursuing – specifically as a Fraunhofer group but also at the University of Passau – is segmentation over time. Imagine you had an object made up of two parts, each of which changes over a certain period of time. You want to separate these parts in order to observe different characteristics depending on the line of inquiry. I like to use a beer as an example: say I want to separate the foam from the liquid so that I can analyze the process by which the foam decomposes. That’s what segmentation is for.
At the same time, segmentation is also part of the compression process: if, for each measurement, I take only the information that’s of interest to me, that might be just 10 percent. That gives me a quick and easy way of massively reducing the amount of data.
AI methods are required for both these processes. Let’s stick with the beer foam example. In this thought experiment, I’m looking to tailor my compression process to the beer foam. Instead of using a manually tailored algorithm that I would then have to reset each time for, say, orange juice or soda, I use a technique that automatically parameterizes itself based on the available data and is therefore suitable for a wide range of applications.
Does data visualization also have a part to play?
Prof. Tomas Sauer: We’ve incorporated a little bit of visualization – you have to have some, otherwise it becomes really hard to understand what you’ve actually done with the data. For large 3D datasets, we’ve already got our own visualization tools and we’ll surely have to think about how we can adapt these for 4D applications. Visualization is obviously also important for end users because they want to be able to interact with the data. Perhaps they’ll want to look at 3D models from a range of different viewpoints. I have to be sure to offer them the option to, say, adjust the observation angle, set cuts, or alter the contrast. So that little bit can actually involve a lot of work.
How much progress has the project made, and when do you expect to see practical applications?
Prof. Tomas Sauer: The project has only just begun. At the moment, all the groups involved in the project are working on generating data, analyzing it, and applying the various methods. It’ll surely be another year before we’ll be able to present our initial results. This technology is basically uncharted territory.
We’re definitely talking here about a research project as opposed to purely an application project. It’s about making things possible that weren’t possible before.
What areas will the technology be put into practice in? What benefits are there for end users?
Ideally, there will be benefits for all end users that work with time-resolved CT processes, for instance by turning over these processes to a synchrotron. Materials science, battery development, biology, and pharmaceuticals could glean new insights. In biology, for instance, this technology could help more precisely analyze cell growth processes. There is a wide range of possible applications.
Are you planning to provide the technology to end users as a finished product, or will you offer it as a service?
Prof. Tomas Sauer: That depends on the use case – sometimes users could do the job themselves with the help of a kit, sometimes they would need us to provide the service. Back to the beer foam: Let’s say the owners of a bar just want to know how long it takes for the foam on a glass of beer to disintegrate to make sure that customers are being served fresh beer. They can get the answer from us as a service provider. But if a museum wants to create an interactive exhibit about how a given material behaves under certain conditions, we’d provide a kit that would allow the client to interact with the technology. At the end of the day, we can tailor what we provide to the needs of the end user.
What is the vision for the project?
Prof. Tomas Sauer: A big win would be if we could lower the barriers to data processing in this field – in other words, remove the need for mainframe computers and expensive software. It’s our job to make this technology readily available to many more users and industries. Our vision is for there to be no technical obstacles to processing, editing, and visualizing data, and extracting the required information. In short, we want to make 4D CT a standard tool in daily operations.
Article by Lucas Westermann, Editor Fraunhofer IIS Magazine
Fraunhofer Institute for Integrated Circuits IIS