Vedran Vujasinović, Setcor: We must ensure that all algorithms are transparent

As artificial intelligence becomes commonplace, the question of security challenges in its use arises. Vedran Vujasinović, director of the Information Security Directorate at Setcor, warns that companies must also be very cautious when using AI.

How should we view artificial intelligence from a security standpoint?

– The biggest security challenge is to ensure that AI models cannot be misused to the detriment of users, while still striving to develop intelligent systems that are vulnerable to the same categories as humans, such as ethics, altruism, and manipulation. For example, artificial intelligence should not provide instructions on how to break into a vehicle, but what if the question is posed in the context of rescuing an endangered child who needs to be urgently taken to the hospital? It is therefore necessary to ensure that AI systems make correct and ethical decisions.

How can we prevent risks in the AI environment, such as the leakage of confidential data?

– Security risks can be viewed from several perspectives. First, employees may inadvertently or intentionally expose the company’s confidential information by entering it into generative AI like ChatGPT, for example, asking it to create a presentation based on a document with confidential information. Second, AI can generate content such as code, images, or documents whose source we do not know, and employees may carelessly integrate that unverified content into their applications, support systems, or production systems.

Third, hackers and criminals can use AI to create perfectly crafted fake phishing emails that can bypass protective mechanisms and reach their target, thus accessing confidential data. This way, attackers without prior experience can use AI for effective attacks, which poses a significant security challenge. Therefore, it is important to take measures to ensure that the data used to train AI models is verified and secure, that algorithms are transparent, that decision-making can be tracked, and that AI models cannot be misused for attacks or unauthorized data collection.

How can generative AI applications be safely adopted?

– Today, many companies want to use AI technology to increase productivity, creativity, and business value, but many forget the risks that uncontrolled access to AI actually carries. For example, can a company allow employees to send secret source code via AI to the internet? Does the company even know that this is happening?

To safely adopt generative AI applications, it is necessary to have an overview of the current state of AI application usage in the company and to make decisions and policies about what is allowed and what is not. If we are already accepting AI, we must ensure that all algorithms are transparent and that it can be tracked how decisions are made in the generative AI application. Finally, when adopting generative AI for internal applications, it is necessary to test and verify the application before use.

Why is zero trust becoming the foundation of cybersecurity?

– For several reasons. Traditional security models are based on perimeter protection, meaning that everything behind the firewall is considered safe within the network. However, this approach is not effective in today’s environment where more and more business applications and data are in the cloud. The zero trust approach, on the other hand, is based on the idea that every activity, whether inside or outside the network, is considered insecure until proven otherwise.

This means that every activity is subject to strict authentication and authorization verification. When combined with checks for malicious code or complete content isolation, the zero trust approach not only increases system security but also enhances flexibility and efficiency while maintaining complete access control and providing employees with a sense of ease in their work.

What security solutions are best for entrepreneurs?

– In the context of generative AI, it inevitably includes solutions for visibility and control of sensitive data, such as Data Loss Prevention (DLP) and Cloud Access Security Broker (CASB) solutions (for example, Broadcom/Symantec). These solutions allow entrepreneurs to see who is using AI services, control who has access to those services, and check all queries to ChatGPT or other AI tools for sensitive content. They can also check images sent to AI using OCR to prevent data leakage in photographs. However, employee security awareness is becoming more important than ever as it will become increasingly difficult to distinguish computer-generated content from authored content.

*Content created in collaboration with Setcor