AI Hub Interview: Saeed Bakhshi Germi on AI and safety

  How did you end up in your current studies? 

Funny story. I think I have one of the most irrelevant backgrounds in the whole computer science department. I started with a bachelor’s in electrical engineering, focusing on instrumentations and computer hardware design. During this time, I got interested in robotics and had a chance to join the robotics team of our university. After working with them, I realized that robotics is super cool! So, I continued with getting a masters in robotics.

Robotics is cool, so let’s continue with robotics

During my studies, I had courses on machine vision and machine learning. I immediately knew that I wanted to continue artificial intelligence specifically, since it was the most challenging part of robotics for me. So, I decided to apply for computer vision and machine learning related positions and with a bit of luck I managed to get into the current position.

When you say that robotics is cool, what do you mean?

The technology is new, but there are a lot of things to consider in robotics. Because I came from the background of electrical engineering, I knew the electrical engineering stuff, but I had no clue the role of mechanics, like the effects of weight distribution on balance. Moreover, I had faint ideas on algorithms and behavior programing for navigation and controlling a robot. These days, there are lots of educational toys, like Lego bricks, to help people understand these concepts easily. Plus, they’re super fun to play with.

I think it’s fun to experiment and see the results of your learning with such tools, and it helps people during the progress.

Where do you find the most interesting development in science? 

Probably anything related to making an artificial human body. Either synthetic limbs, organs, or even a map of brain!

Can we make a human brain somehow? Can we upload our consciousness on a computer?

What have you been up to in 2021?

At the start of the year, we realized there is no real previous work done in the topic of safety on AI. At least not on a practical level. We realized that there is a missing link somewhere in between. That companies want something practical, whereas academia wants something new. While academia claims that we have working tools from years ago, industry can’t use them due to practical and legal issues (not having proper standards for verification and validation).

We approached the problem by preparing a list of all the possible faults that can happen within an AI system. Then, we started targeting specific faults in separate projects, and tried to present a suitable method of dealing with them. While the solution might not be what the industry wants now, It can arrive there with more time and research.

What about the tools you have been using? What algorithms or other tools have you used the most? What would you rank on your top list?

Well, as an engineer I use pretty much any relevant engineering software and tool. I have been using Python programming language but not in the same way as a developer might do. I don’t need to understand the fundamentals of software programing (like object-oriented methods or so) since I only use Python as a super complex calculator to simulate my model.

Due to the open-source policy in computing science field, we have access to all previous codes and algorithms of anyone and everyone. We also have huge companies that create decent algorithms, like Facebook. They all participate in this. What we do is basically picking up the latest algorithm from Facebook or Google and adjust it to suit our demands. There are also some frameworks like TensorFlow and PyTorch.

We don’t really need to code that much. We just need to use the Python to run our codes.

Is this time consuming?

Finding material is not. However, you need to spend a decent amount of time to understand how the code works, as they come in a big package with tons of files that refer to each other. After a couple of times, you get used to how to read them, so it becomes easier.

Using the codes also is time-consuming, as the training sessions may be days or even weeks long depending on the task, and you don’t have any sense of how the system works until you have a few training sessions. I remember my last paper where I had to run a code for two months straight to fill a table with 12 numbers.

I had to run codes for two months straight!

That takes a lot of computational power and must be energy consuming.

Is it good to use this amount of computational power for safety? My answer is yes.

Eventually, the computational power for training a model and implementing it is negligible when compared to the tons of energy used by the main heavy machinery that the algorithm is intended for.

 

What problems and threats do you think companies face with AI safety the most right now? Have you heard any examples?

Right now, most problems probably relate to creating datasets. This is due to a lack of experts in the companies, the huge cost of gathering a proper dataset, and the competition between companies that restricts the open-source practice.

Another issue would be the lack of a proper standard for AI in safety-critical applications. These standards take a lot of time for development, and they are already left far behind the technology. Moreover, even the existing standards do not guarantee safety, as a simple system can be purposefully designed according to all the criteria and still fail and cost resources.

Just having a standard doesn’t mean that your device is safe. It just means that your device has followed specific rule sets.

You mentioned that the companies might not have the right quality or amount of data they need, which is a problem. Do you think the university could be the solution to that? Because the university has a lot of practice data sets.

The university on its own, probably no. However, there are some organizations like VTT, FIMA, and AI Hub that fill the gap between the university and the industry.

Due to the competitive nature of companies, sharing information between them is probably not wise. On the other hand, university is too focused on the idea of innovation and does not realize the applications of a technology is an innovation itself. An organization that does not suffer from these issues is the ideal solution for solving the main problem!

Why would anyone share their dataset for free? Even selling it is not always an option.

How many people graduate on AI and safety? Are there enough people to cover the need?

We probably have enough people in the AI field. Typically, those who study AI are not familiar with safety-critical applications and requirements, and those who work in safety domain do not have expertise with AI solutions. The main issue is the huge gap between these two fields that never interfered with each other. So, we need people that understand both sides.

DSII helps this process by training doctorates for industry. As with any other problem, the first step for solving this is finding the motivation and funding. Those come from big corporations and government, which requires the society to understand the benefits of the proposed solutions. With the increase in the popularity of autonomous vehicles, maybe we will get there faster than we expected.

However, when you involve society, you create another problem. The age of information also brings misinformation with it. Having a piece of technology in a closed environemnt within a company might not trigger anything in the society. But once you get within the society then the ethics, logistics and liabilities become relevant problems.

When you involve society, you create another problem.

What are your primary AI safety recommendations? Let’s say if I was running a company and I wanted to develop AI safety in my company, what should I do?

With the current limitations, I would recommend separating AI from safety. We have working safety systems and there is no need for an immediate upgrade on them. Meanwhile, we always need more performance from the overall system.

AI can provide the only viable solutions for some problems.

As an example, let’s say you have a car with a safety system (like a distance measurement sensor) that can guarantee the vehicle won’t hit any objects within a 2-meter distance. Now if we add a camera and use object detection, we might be able to provide information about obstacles ahead of time, to enhance the overall performance and smoothness of driving the car. In this case even if a detection is failed the system is safe with the initial sensor, but when the detection works it is providing additional information for path planning.

AI can be used as a decision support system instead of a decision-making system.

There are very many problems right now, so you can use the AI in the planning stage. It does not rely on safety there. I think  it’s better to use the AI in pure planning stage or like as a decision support system, not decision making system.

Earlier, when you said you need better output, what do you mean?

Some problems are impossible or too complex to solve without AI. Let’s give an example with a cargo ship. The vessel must move to several ports, load, and unload multiple containers, while maintaining the balance of the vessel and optimizing to minimize the amount of useless container movement. This is a big numerical nightmare! Lots of variables, non-linear relationships, and no known formula. This is what AI is built for! it is good at doing heuristic numerical optimization with large amount of data.

Now in this problem, it is possible to solve the issue without AI, but it is too tedious and time consuming. So, using an AI solution would result in an increase in performance and more profit.

Juggling numbers around is what AI is good for

So you can use AI there for, so it will provide you with better result, which means you will have to move less cargo around, which means saving time saving money, faster shipment.

What are your current goals when it comes to your studies? Where would you like to go from here in the next 1 to 3 years?

I have one year remaining in my studies now. The goal is to finish my last papers and prepare a draft of my thesis by the end of summer. This way, I’ll have some free time to find a job. In my thesis, I’d like to have some answers for the questions asked here and in the studies. I won’t be able to answer all existing questions, as it requires more than a PhD to do so, but I will do my best. The plan is to continue working on these problems from industries if I get to join one of them.

This topic is new, challenging, and it intrigues me! I’m having a fun time learning the details of the work and getting to meet like-minded people.

How do you spend your free time?

My work/hobby schedule is weird. There are days that I have extra energy, and I work 14 or more hours a day. There are also days that I feel a bit tired and might not be able to make it past a few hours of work. Typically, it all balances in a longer period (a month or so). As for my free time, I spend them on watching some documentaries, live streams (mostly old-school D&D games) and play video games (which is on pause for a while now).

Thank you for the interview!

Interview by Ritva Savonsaari / AI Hub Tampere in December 2021

Image sources: Saeed Bakhshi Germi / Slide set on FCAI.fi

• Blog
• Yleinen