Software Security: Catching Bugs Before They Catch You

Olivier Colson: 0:00

Even when you think you're using a password that is super smart, it's actually something that is very likely to be chosen by other humans. Oh, the CEO is not very technical. He doesn't understand anything about Odoo. It's a lot of screens and mumbo jumbo that he doesn't understand, but he wants to be superadmin. And then guess what? One day he's going to make a big mistake and maybe delete some important data or cause massive issues in the system because he doesn't understand and doesn't really need those screens. There's one setting that definitely everybody should be using is the second factor authentication or two factor authentication. You also have to ask yourself, what could possibly go wrong? You have to take the kind of the opposite side and say, okay, let's imagine I'm an attacker and I want to break on purpose the software. I'm like trying to destroy this bridge.

Olivier Dony: 0:46

Hi Odooers, and welcome to another tech and dev episode of Planet Odoo. Today we will explore one of the most critical aspects of software development security with security and platform expert Olivier Dony. We will dive into Odo's philosophy towards security and how it's much more than just protecting your platform. From cloud management to self-hosting and best practices to avoid cyber attacks, Olivier will share with us key considerations and best practices for ensuring that your Odoo installation is secure and well protected. Whether you're a software developer, a business owner, or simply interested in learning more about software security, this episode is for you, so lock the door, turn on the alarm check, there's no one at the window. And let's get started.

Olivier Colson: 1:28

Hello, Olivier.

Olivier Dony: 1:29

Hello, Olivier.

Olivier Colson: 1:30

Nice to have you here. So could you explain who you are and what you're doing at Odoo?

Olivier Dony: 1:35

Yes. So I'm one of the most ancient employees, if I may use the word.

Olivier Colson: 1:41

Another Odoonosaur.

Olivier Dony: 1:43

Yeah odoonosaur sounds nice

Olivier Colson: 1:46

And so what are you doing?

Olivier Dony: 1:47

So I initially joined Odoo as a developer back in 2009. It was really a long time ago. The company was very small at the time, actually. I think it was called Tiny still just rename I think the software open ERP. Before that it was called tiny ERP. I always have trouble remembering exactly when we switched from one to the other. So I cheat and I go to our download page for the nightly builds where we still have the old versions from history and back from version three. And I can see when it was renamed based on the file names in the downloads. So I joined at that time in initially as a developer, but the first demand we had was for services and training customers and partners.

Olivier Colson: 2:37

Yeah, it was the time where everybody was doing a bit of everything.

Olivier Dony: 2:41

Yes, yes. We were like 20 people I think, in Belgium. So yeah, everybody was doing everything. And it was funny, the onboarding was like two weeks, classical for a developer in the company. One week of functional training on the software, then one week of technical training. And I think 2 or 3 weeks later I was scheduled to give the trainings myself, of course, and then it was kind of a interesting time. And for the first functional training it was like reading the source code of the software the night before giving that same topic to the partners the next day. So I was discovering the logic and the bugs as I was going. It was pretty funny.

Olivier Colson: 3:30

A lot of improvisation, but I mean, it has to be like that at the beginning, right

Olivier Dony: 3:34

Yes. Yes.

Olivier Colson: 3:35

And so how did it evolve then? So it's not what you're doing now anymore. You're not improvising anymore, right?

Olivier Dony: 3:40

Right, but a bit less. We're still doing a lot of improvisations. That's kind of the company culture, right? Not overthink matters. And, well, now I'm doing several things, but I'm mostly concentrating on security and management of our platforms, so the cloud.

Olivier Colson: 4:03

How did you get to that in all that period where everybody was improvising? Uh, where was the moment where people were like, Oh, you're going to be the one in charge of of security, or you're going to at least have a look at this. Because I guess at the beginning, when you have few people like that in the company, it's not really the core focus, right? It's at some point someone realizes that it's important and puts it forward, right?

Olivier Dony: 4:24

Yes. And actually, when we're starting doing, let's say, security some sort of seriously, it was in 2013. And if you think it was quite a different time back then, I mean, ten years ago in terms of security, there was a lot less awareness. There were a lot less big hacks in the news,

Olivier Colson: 4:45

A lot less danger as well.

Olivier Dony: 4:47

Yes, a lot less danger. Basically, the Internet was a less dangerous place or slightly so at least. And I was kind of interested myself in these security topics. So there was nobody actually doing that in the company, obviously with our size. And we're starting to notice that some bugs had security implications. And I've always been a bit obsessed by details and having kind of a complete understanding of the low level details. So, for instance, I'm the guy who tries to understand how Encodings works in in programming languages and bytes and characters, how you convert them back and forth. And by that time nobody was caring much about that. And security was one of these topics where you have to kind of dig deep in the code to understand how a normal bug can turn into a security risk. And we started to work with the community. We also had this growing community, and one of my roles as a bit of everybody was doing a bit of that, but I was kind of also community manager and I was interacting a lot with our open source communities on our platforms, on our projects, and we started to have people with security minded focus reporting bugs to us. And then I thought, well, actually we should be doing what these other big companies are doing and to make a kind of a formal security program to be first to make it like explicit that we care about this, and also to welcome the security researchers, because it's often something that they complain about, is they have a really hard time contacting security representatives or getting in touch with the companies.

Olivier Colson: 6:29

On this kind of topic. The more reactive you are, the better, because I guess you want people to know that they can report and to encourage them doing so because it's better to have more than not having enough, I guess.

Olivier Dony: 6:40

Definitely. And you want to be listening for them so that you have the best chance of catching your bugs before the bad guys catch them. We definitely want to encourage those white hat people. So that's when we decided, well, okay, let's go. I'll just invent a program and write down some rules and put a page on our website. And from then on we started to get a slowly increasing flow of security reports.

Olivier Colson: 7:08

And it improved over the years to reach the point where we are now. And I guess it's still improving now, right? And it's still improving. Never. You're never done with this kind of thing?

Olivier Dony: 7:16

No, it's it's it's always increasing. And the size of the team, fortunately, has increased as well to handle this influx of lots of topics in security. Definitely. And the responsibilities of the security team have grown. So, more people are now doing that full time. And we have in the scope of the security team, we don't only have the security of the product, but we also have the security of the platforms because now we have hosted platforms where we have customers on the site.

Olivier Colson: 7:48

I suggest we go back on the the the technical aspects and how Odoo handles that internally a bit later. But first, let's imagine someone is configuring a deployment of Odoo. What can they do from a purely functional point of view to to make it secure? Because I guess there are things that they could do badly and that would be security risks, right?

Olivier Dony: 8:12

Definitely. It's a very good point. It's something that I wish more people were aware of when they start using Odoo. So, the first thing they have to realize is that there is a security philosophy in Odoo and they have to understand how it works to decide what they want to do with it. So the most obvious thing that you notice when you start Odoo is you don't get a ton of configuration screens and options that you have to fine tune to get started with the desktop, right? It's a big focus of our usability, is to make it frictionless as much as possible and to remove all the steps that get in the way when you try to discover how the software works and to understand and see if it fits your purpose and that comes to the kind of the detriment of security because it means that you have no chance to take security decisions at that point. So once you pass that that initial phase of discovering the software and you decide that, okay, this is what I want to use and I know what apps I'm going to install, then it's time to delve into these security topics at that point. Because basically, to allow this discovery like a boundless discovery of the software, we make it so that the initial settings are very open, everybody has full access to everything. And if you start to invite colleagues to show them the system and to try them to have them try it.

Olivier Colson: 9:43

Have full access by default. Yeah, exactly. They are like yourself, their full admin or close to full admin. So, when the time comes to maybe invite more users and if you have a company that has a certain size and not just a single person company, but you have more participants, more employees. Or maybe sometimes you will have interns joining the company for a short time. You definitely want to restrict the security access. And so, at that point, you go and look at what we call these role based access control system that we use in Odoo, where each application, if you go into the security settings of the users, you can see for each application what level of access they have. And there is this dropdown system where you you choose the level of access for each app and it's a hierarchy of access. So basically, each level inherits from the lesser levels, like a sales user has less power than a sales manager and the sales manager has all of the powers that the sales user has. So that's, that's the basic of it. So first you will need to decide, okay, what access do I give to whom in the company? And you need to understand what those accesses correspond to as well. I guess because it might not always be obvious, Right?

Olivier Dony: 11:04

Right. Because it's kind of okay, what does it mean to be a sales user or to be a sales manager? What does this open access to?

Olivier Colson: 11:12

And I guess you also have to make sure that you understand all the different features that you can configure in the applications you installed and make sure that you only enables the one you want. Because it's not just access rights, it's also the settings of Odoo itself, right? Because I don't know, I don't have an example in mind, but you might have something that could be enabled that you don't know about that is enabled by default, is there and creates a risk because, well, whatever reason and you didn't configure it and you should have. Right?

Olivier Dony: 11:45

Right. Definitely. So there are settings that allow, let's say, very powerful access to web developers. So someone may need that or someone may not need that. So you have you have to have a look at all the different settings and see, okay, I want my users to be able to edit the website or not, and based on that you will give them the access that they need. There are other settings that may be less obvious, like being able to edit mail templates is something that has quite a lot of power and by default it's turned on and it's not obvious because it's not an app. So, it's not something that you will immediately think about, but there is a setting where you can decide, okay, do I want to restrict access to mail templates or not? And you can find it in the configuration screen. And then based on that, the proper access will be granted or not to your users.

Olivier Colson: 12:39

It's interesting because when it comes to security, a lot of people seem to to think that it's just protecting yourself against the outer world and people outside of your company or your group of people that would like, you know, pirates on their ships that want to attack your fortress, whatever. And here we are talking about internal risks, actually, of a user having more rights that he should. And actually, it's also external because if his account is compromised, the attacker will get access to more things that you would have otherwise. Right?

Olivier Dony: 13:09

Right. That's one of the core principles of security, is trying to implement the least required privilege that your users need so that indeed it will prevent them from making mistakes. Let's say everybody is an admin."Oh, the CEO is not very technical. He doesn't understand anything about Odoo. It's a lot of screens and mumbo jumbo that he doesn't understand, but he wants to be super admin." And then guess what? One day he's going to make a big mistake and maybe delete some important data or cause massive issues in the system because it doesn't understand and doesn't really need those screens.

Olivier Colson: 13:44

Indeed, one might think, how could this happen? But actually it can happen because as you were saying, in Odoo, the UI is as clean as possible and it tends to minimize the number of clicks that a user has to do. But sometimes it comes to the price that there is a bit more magic going on behind the scenes and operations that will happen because you clicked on something. And if you don't know what this thing really is and I'll just clicking on everything and hoping it will work, you'll get in troubles.

Olivier Dony: 14:11

Definitely, and particularly in the area of settings configuration settings where a single click that looks just like a checkbox. It looks harmless, you just checked an option, right? But then it triggers, as I was saying, this system where people can edit mail templates massively or it changes the number of apps that are installed behind the scenes required new roles.

Olivier Colson: 14:35

And new rights and other issues like that. Right?

Olivier Dony: 14:38

So lots of like consequences that you may not realize immediately when you're changing just an app.

Olivier Colson: 14:45

So about, uh, what I was saying about, uh, accounts Compromission what can they do to protect themselves against that?

Olivier Dony: 14:53

Oh, there's one setting that definitely everybody should be using is the second factor authentication or two factor authentication. It's something that we have since version 14.

Olivier Colson: 15:05

How does it work for people that don't know it? Because it's available in a lot of platforms, a bit everywhere, but what is it exactly? What's the idea behind it?

Olivier Dony: 15:13

So the idea is instead of relying on a single protection to protect your account, basically your password, which is something you know, you are also protecting your account with a second factor of authentication that is normally not something you know, but here, for instance, something you possess It could be also biometric, then it's something you are instead of something you possess. In the case of Odoo, by default, we support second factor authentication with something you have, and that typically would be an authenticator device that produces codes. You know those six digit codes that you may be required to enter when you go on Gmail, for instance, or you may receive them by SMS on some platforms, it's pretty similar to what you have in Europe when you want to access your bank account as well, You receive like a device that you have to tap a PIN code and then it gives you a series of digits. So basically it's something like that and Odoo supports it out of the box, but you have to enable it. Each user has to take that step and go in the security preferences, start the process of activating two factor authentication, and for that they will need an authenticator device. So, typically an app on their mobile phone. There is the famous Google authenticator, but there are also many other apps and many password managers, which is another tool that many people should start using. Many password managers have a feature to provide these two factor authentication. So, you scan a QR code and your authenticator learns the secret to be able to generate those series of digits. And it may sound something like a burden, it's going to be annoying because every time you have to log in now you have to take out your phone and look at those six digits and type them and it feels it could be annoying for users.

Olivier Colson: 17:13

Yeah, but I mean, it's kind of the necessary price because here to get into your account, someone would basically need to have your phone and your password, which is unlikely to happen, or at least way more difficult than just guessing your password because you put something that is just the birth date of some singer you like or things like that. Because it happens all the time. You know, over years we developers kept on saying to people, don't use strong passwords and a lot of people still don't get the thing. And I guess here it's more automatic and, and it's just easier in the end because the responsibility is not only on the password and on finding something that nobody will think about. It's not just someone thinking and guessing the password behind the computer. It's programs with lists of things to try, and sometimes they are smarter than you. Actually, that's a problem.

Olivier Dony: 18:06

Yes, you're right. Humans are known to be very bad at choosing passwords because we are human, so we are very predictable. And even when you think you've chosen a password that is super smart, it's actually something that is very likely to be chosen by other humans and the space of passwords to try for what we call brute force attacks where you just have a program, try all the possible passwords until you get in. It's actually small. But that's not actually the most common way that accounts get breached in web applications in general.

Olivier Colson: 18:43

What's the most common way, then?

Olivier Dony: 18:44

Well, the most common way is because the password has been leaked somehow. And there are many ways for it to be leaked. The most obvious one is if you use the same password on different platforms and one of these platforms gets hacked and their passwords are stolen, then it's very natural for the the attackers to just take those passwords and try them on other places where you are likely to have account.

Olivier Colson: 19:06

Because again, we all have that friend who some some day forgot his password and something you were sharing with him and it was like, oh, wait, I'm going to try my usual passwords. And so actually he's using like a set of 3 or 4 passwords on everything. So basically, if one of them is compromised, uh, it's done.

Olivier Dony: 19:22

Exactly. It's very typical. So that's, that's one case. And of course the best way is to have a password manager and just copy paste your password and then you only it's like one password to rule them all. You just need the master password of your password manager and everything else is randomly generated. So that's the ideal world. But then if you're not in that ideal situation, having a two factor authentication helps because then even if your password is compromised somewhere else, you have the second factor that is supposedly going to stop the attacker. Of course, that's still not totally bulletproof because you could be fished. And now the fishing systems were attackers give you a fake login form and then they record what you're typing there. They use this trick to steal your password, but now they're more sophisticated and sometimes they also steal your two factor code. And those six digits are valid for 30 seconds.

Olivier Colson: 20:27

Just just to recall phishing.

Basically, you receive that mail asking you: 20:30

'hey, could you reset your password? Because we got something that needs you to do this action or that action' and it's actually redirecting you to something that is not at all the right website your thing, but you think that it's legit. You do it because you're a bit naive. And then that's done. I'm explaining it in a very stupid way, but of course it can be done in a way more intelligent way and be less way less obvious to people. And then you screw it because you're giving your password to someone else.

Olivier Dony: 20:57

Exactly. And that's why we have all these phishing awareness campaigns in most companies where we try to teach people to be wary of these strange emails, to always verify the address bar of the browser to be sure that they are entering their credentials on the actual website where they're supposed to enter it to be very careful. If they're suddenly receiving an email that asks them to go to some place they're not used to go. So, to be careful at every step, whenever it's about credentials or anything sensitive in the company, even like payments like definitely if you receive a message that tells you 'Oh, the bank account for paying my invoices has changed. This is the new one.' Well, you better be suspicious and say 'okay, let's call the person on the phone to double check that they actually are changing and they just haven't simply have their mailbox compromised' which is a very common system for attackers. They run these phishing campaigns where they compromise a single mailbox, but then they use that mailbox to contact the other people that this person knows. And they do it very successfully because they are sending from the actual email of the person. So and they continue like that. It's like a wave that expands.

Olivier Colson: 22:18

And let's now imagine that our database falls because of one of these attacks. And someone manages to enter the database and locks the, let's say, the access to everyone and ask €10,000 so that we can get back into it and continue working. What can we do against that and to when it happens to handle the issue and to survive without paying the guy of course, what can we do for that?

Olivier Dony: 22:44

So definitely you've taken all the steps that we were thinking about. You made users aware of these issues. You have them set up two factor authentication, but still, sometimes it happens. They are very sophisticated attackers and very focused that could get into your system. And then they will try to delete your database and tell you, okay, I'm not going to give you back your data unless I'm paying.

Olivier Colson: 23:09

About logging, but it could basically just be sabotage and and deleting stuff so that they are sure you can't get away with it. Uh, and your database is just broken. It's over if you don't have a solution.

Olivier Dony: 23:19

Exactly. And for that, the best solution is, of course, to always have a backup that is safe and from where you can recover your work so that, if the worst happens and nowadays it's more like when is going to happen rather than if it's going to happen. Because you see in the news, it's every day even the largest corporations with lots of money to defend against those attacks. Everybody can be a victim of that any day. So you definitely want to have backups that are secure and to be secure, you want them to be out of reach of the attacker because they can be very thorough when they are attacking you and they want to extract money from you. They want to force you to pay because you have no choice. And so, if you have backups that are available to them, they are going to destroy them as well. So what you want is to have backups that are protected. There are different ways to do that. Well, for example, what we call air gaps is basically they are disconnected from the Internet. So the attacker will definitely get into your system through the Internet. And if you have a backup that is on a disk that is not connected to any computer and is just sitting in a safe box.

Olivier Colson: 24:32

So it can't be attacked. I mean, not like this at least. They have to enter your parent's house and steal it.

Olivier Dony: 24:39

Exactly. That's definitely the way you can protect it. There are other systems in cloud based backups where you can set up policies that say, okay, your this backup is frozen and even yourself, you are not able to modify it or delete it.

Olivier Colson: 24:55

So even if they get full admin access, they will not be able to delete that.

Olivier Dony: 24:58

Right. Those are frozen systems that are designed to withstand attacks like ransomware and other very focused attackers who would want to destroy completely your company.

Olivier Colson: 25:12

And then I guess those backups also need to be encrypted so that if something happens on the surface when you're hosting them, basically, well, they get something encrypted and they can't use it unless they manage to get the key, of course. But that's one more step and making it more difficult for them.

Olivier Dony: 25:29

Right. Definitely. State of the art security would require that you also encrypt your backups. That's definitely, for instance, what we do for our cloud platforms. We encrypt everything when it's stored on a disk or when it's transferred on a network so that if anybody is able to intercept that, they're not going to be able to extract the data. And of course, that's also a liability because if you encrypt everything, it means that your key becomes very important. And if you lose the key or if the attacker is able to destroy the key, you have the same problem. You have some data, but you cannot read it. So it's locked by yourself, not by the attacker, but it's still kind of.

Olivier Colson: 26:12

So it's interesting because every security step you take brings you to new security steps you need to take to secure the thing you just introduced for the previous one. So it's an endless line of measures that you need to take, I guess.

Olivier Dony: 26:25

Right. It's it's really a series of doors that you have to put to protect your most important assets. And yeah, it's really difficult.

Olivier Colson: 26:36

So Odoo offers the ability to well deploy your database on Odoo server directly. So with the SaaS and with Odoo, but you can choose to just do it on your side and deploy it yourself. Is it something that you should do actually, because you're explaining us that it's very complex? Is it worth it to do a self deployment like this? Would you say it's something recommended?

Olivier Dony: 26:58

Actually, I would definitely not recommend it by default for everybody. It's a complicated step. It's complicated on the security side, and in 2023, it's becoming very important to take care of that aspect. Otherwise you're just a victim asking for an attack. But then there are many more things to cover than just security. There is a performance, there is integration of the technology with different systems like email. You have incoming and outgoing emails that you have to integrate with Odoo. So there are lots of little things that are not easy to configure properly when you're just starting. So, it's kind of a full time job to administer an Odoo installation. We make it seem very easy if you just download the installer and just click, in two clicks you have it running on your computer and that's perfectly fine for a demo or for trying the software. But if you want to really run your business with Odoo, you have to run it properly with all these integration setup, with all the security setup. And that's far from trivial. We have a big documentation for that.

And if you read the deployment guide, there are lots of steps: 28:06

you have to configure your web server, you have to deploy HTTPS certificates because nowadays on the Internet, if you don't have SSL https encryption on your website, your website is unreachable, basically. Then you have to properly configure the storage for your sessions and for the files that you are going to store in the database. Then you have to secure against denial of service attacks and make sure you cannot your site cannot be taken down by an attacker. So, there are many complex topics to know until you can get a safe deployment.

Olivier Colson: 28:49

So the message would be if you if you really need to deploy on your side, on your server, you need to make sure what you're doing before starting doing it, right?

Olivier Dony: 28:59

Yes. Really, you need to make sure that you have someone with the skills or you take the time to develop those skills yourself, to be able to self-host your system, at least if it's to run any kind of business. And it contains sensitive data.

Olivier Colson: 29:15

Yeah, it's for the cases where you really need. Otherwise I guess it's always better to just rely on well, if the provider is trusted, but if it's not, why are you using his software? So to rely on the providers platform and for us the SaaS and just say, okay, let's delegate that to them because they know how it works, because they have like thousands of customers already running like this. And so they are protected by the this team.

Olivier Dony: 29:41

So you have all these different aspects, and there is a real case where you may want to self-host your your Odoo deployment is, if you have some legal requirement that says, okay, the data cannot be outside of my country. And if Odoo doesn't have a hosted system in that country, then you have no solution. But even then, my advice would be to go to some professional hoster for Odoo. We have a very big network of partners and those partners are more experienced with all these caveats of properly deploying a modern web applications like like Odoo. And even for partners, it's not an easy job, actually. You have to look for experienced partners. That actually is one of the reasons why we started Odoo.sh, which is a platform as a service, and it's really dedicated to partners who want to develop solutions for their customers, but without having to take care of all the deployment aspects and all the security aspects, at least on the platform level.

Olivier Colson: 30:45

And for newcomers among the partners, it be might also be the good way to learn it and get the thing. And eventually maybe if it's legal and they need to deploy some servers on their side. But knowing already the product and what could be needed for it.

Olivier Dony: 31:03

Mm.

Olivier Colson: 31:04

Okay. Is there something else you would like to mention about how to secure an Odoo deployment? Functionally speaking, before we go on.

Olivier Dony: 31:11

Maybe one aspect that may also be interesting for new users of Odoo is the option to install external applications. We have this big app store that is very interesting with thousands and thousands of apps and new ones being added every month. So it's very tempting to go there on apps.odoo.com and to try to install random apps that you feel compelled to try. And it's very tempting to go on the app store and to try and download some apps. But you have to be careful, because we don't currently have a review process for these apps. So you're basically trusting the developer of those apps to not do anything bad to your data.

Olivier Colson: 31:58

The code of the code of the app is not endorsed by Odoo itself. It's just we are offering the the right to download them from the app store, but not developing them ourselves nor checking them at all. Right?

Olivier Dony: 32:09

Right, exactly. So anybody can publish basically an app there. And as we were talking about security, it could be that someone would develop a malicious application that will actually create this breach and encrypt your data and try to run some you. So obviously, we would take that down immediately as soon as we were aware of that. But currently we don't have a systematic review of all the apps. So it's one of the areas that definitely I would like to improve and to work on at Odoo. And we have some ideas to improve this situation. So we may have some new developments coming up on that front. But the advice would be really if you want to install something from the App store, look at the reviews, look at the provider. There are some trusted providers like the OCA, the Odoo Community Association, that maintains hundreds of add ons, hundreds of apps and that could be a way to trust some providers. But you have to be careful and maybe ask someone technical to review the application before installing it on or downloading it for for your own database. Okay.

Olivier Colson: 33:23

And since we're talking about the technical side of things a little bit, what would be the typical problems you could have as a developer? So let's say I'm doing my deployment and I want to do some development myself on it or have one of my developers do it. What do I need to take into consideration? What are the risks exactly?

Olivier Dony: 33:43

Lots of risks there.

Olivier Colson: 33:45

Way more of things.

Olivier Dony: 33:47

That's basically what we do all day with the security team, is reviewing the work of our developers and improving the tools that they use to make sure that we catch all those mistakes early on. I like this story of comparing software development and the security of software with other engineering problems like building bridges

Olivier Colson: 34:14

It's always bridges, when you talk about engineering problems, people are always like, Oh, let's build a bridge. It's the typical example, sorry, bridges.

Olivier Dony: 34:21

Or cars, or girls work as well. But let's say bridges, when you're designing a bridge, you know the hazards that it's going to have to withstand. So, you know, the the weight of the the cars that are going to pass over. You can imagine what is the largest and the biggest quantity of cars that could be on top of the bridge. You know, that it's going to have to withstand the wind. And you have to you can you can look at the statistics of the wind. You can imagine the magnitude of of earthquakes that it has to withstand and all those different risks that the bridge has to be able to resist to. And once you factor all of that in, you have a very solid bridge. But for designing a software, it's a bit different because all the hazards could be combined at once. It's like if you were a bridge and suddenly the wind, the earth, so everybody wants to attack you at the same time because they have a purpose, that they want to destroy the bridge, right? They can even have a big trucks with acid spilling on the on the bridge and everything at the same time. And it's impossible for a bridge to resist to all of that at the same time. It's the same thing for software, it's going to be an attacker really focused on trying to break the attacker could combine different vulnerabilities to get a bigger effect.

Olivier Colson: 35:49

That's a big difference, actually, is that the attacker is trying to get the bridge down or all those catastrophes that you're talking about are very serious and very dangerous. But they are not trying to destroy the the bridge. They do it by accident. So that's a big difference.

Olivier Dony: 36:02

Exactly. That's the main point. And so, it makes for a very different mindset when you want to secure software development practices. So we have a big list of all the security pitfalls that developers fall into frequently. And maybe I can describe some of the top three problems that we find when we do security code reviews. The largest category of issues is based on injections. Injections, is when you are basically combining two different pieces that do not fit together, and I'm talking mainly of data and code, so when you're building an application you write code, but the code is is pointless if it doesn't do anything and what it's supposed to do.

Olivier Colson: 36:57

Data to work with.

Olivier Dony: 36:58

Exactly what it's supposed to do is to move data around. And that's what Odoo is,a big thing moving data in the database, more or less. And so, as a developer, it's something that you have to do every day, is to take data and process it. And when you are processing data, you mix the data with code that you wrote. But these two are very different beasts. Basically, your code is something that you are orchestrating very carefully, fine tuning and is going to do exactly what you want. But data is something that comes from outside and possibly from the users, but also from attackers. And it generally is tempting to look at it as something that is harmless, but it can become a wild beast and attack you from behind if you're not careful. So that's kind of the dangerous, the principle of of injections. So this is the biggest class of issues that we find. And there are different variations of injection attacks. The most common one we see is what we call cross-site scripting attacks. So abbreviated XSS because there are lots of jargon and abbreviations in the security world. And that typically happens when you manage to have the application execute arbitrary logic that was not part of the code initially, and inside the browser. So the browser is running the client side of Odoo with JavaScript. And, once some developer has not been very careful while processing this wild beast of a piece of data and has injected it into the code that is generated by the application, then the attacker controls the output in the browser and is able to execute arbitrary logic. And when doing that they can do anything. They can escalate their privileges, they can create an admin account for themselves and an access attack is generally triggered by clicking on a link or visiting a page that the attacker has prepared a malicious link that is sent by email, for instance. So another reason to be very careful with suspicious emails.

Olivier Colson: 39:18

Okay. Other type of error?

Olivier Dony: 39:20

Yes. Another type of error that is quite common is the checking of permissions. So when you use the framework, the Odoo framework, they are permission controls built in into the data access layer. So whenever you want to read data or modify data, you have to ask for the permission and the system is doing that automatically whenever you you are working with data. This is by default and this is mapped to the role based access control definition that you define as the administrator as we were discussing. Sometimes the developers they need to perform an action that the normal permissions wouldn't allow. Typically when they're rendering pages for an anonymous visitor that is accessing the public part of the website, at that point, you don't have any context where you know the user and you can identify the roles that they are granted. So you have to go into a special superuser mode. We call that pseudo. And when you do that, there is no access control anymore. So you have to be very careful what operations you are going to conduct in this special mode. And that's another common place where we find security errors in the code, is when you make this pseudo context a bit too big and you keep it for too long and then you are performing extra operations and sometimes that can become dangerous because the visitor may be able to pass different parameters to that special page that you are rendering in pseudo mode

Olivier Colson: 40:59

Things you wouldn't have access to normally.

Olivier Dony: 41:00

Right. And they can control that normally the security of Odoo is not managed by hiding things like links and menus. The security of Odoo is really controlled by the permissions that we defined for each role. But when you are using this pseudo mode, you are bypassing that completely. And something that normally is impossible, like being able to play with the URL and changing the ID parameter in the URL, you know, you just change digits in the URL bar and you suddenly are able to access the things that you're not supposed to have. That's normally impossible in Odoo because of this imprinted access control in the data layer of the framework.

Olivier Colson: 41:42

So, for example,you don't have the right to access the accounting and you try to you access something you have access to. So a project task, whatever you change in the URL to say, Oh no, but actually I'm not accessing project task. I'm accessing account move and giving whatever ID and trying to access that. Normally in standard use of Odoo, you do that, it will not work because it will check the access. Right? When you try accessing the the link and it will tell you a dude you can do that. But here it's different. So.

Olivier Dony: 42:10

Exactly. Because if that special situation occurs and you are using this special pseudo mode, then it may be possible for the attacker or the visitor to play with the parameters. And you have to be extra careful about extra careful about that. So that's another common one. Other areas in terms of permission control are having private methods be public when they should be private. So, those are special low level methods that are not supposed to be externally callable. And if you are not careful when defining them and you make them public, they may be also exposed to attackers when they shouldn't be. And if that special method, let's say, is immediately performing a task in pseudo mode, in superuser mode, then that immediately becomes a liability and a danger.

Olivier Colson: 43:01

So basically everything that does not need to be public should be private.

Olivier Dony: 43:05

Exactly. As you see, there are lots of topics like that where you have to be kind of careful all the time when you are designing.

Olivier Colson: 43:14

And very strict about about thing because one could think, I mean, when you're at university, you're learning how to to program and and you learn about those things about private public stuff you you get the idea like okay it's fine but in the end in the thing you need to program for university. You don't really care about that because well, you don't care about security. It's just a program for the course. And it's only when you get into the real world that these things become critical actually, or can become critical and that you need to be more strict about them. So I guess, indeed, maybe too many people don't realize that immediately. I need to be to be told about that.

Olivier Dony: 43:51

Yes. And and it's a key point that that you you are mentioning. It's something we wish wouldn't be the case. We would like the software and the software to be easy to use. And the framework to be easy to use and to be impossible to shoot yourself in the foot like that. But it's something where you have to have some basic awareness of the risks.

Olivier Colson: 44:14

Okay. Any other error you would like to to talk about? Yes.

Olivier Dony: 44:18

So another important class of permission problems that we have is called the remote code executions. Rc Abbreviated. And again, it's a kind of injection problem where this time, instead of crafting HTML or HTML for the browser, you're crafting a command that you're going to pass to the operating system to, let's say, produce a PDF file or render some image or anything that you're not doing in Python directly in the logic of your app. And at that point you are again going to combine some data and some code to produce the command that you are executing. And developers have to be very careful about that as well, also when they are importing Python libraries to produce those commands, there are always cases where it's possible to bear the difference between data and code. And every time when you want to transform data into code and to mix both, you are going to have an escaping or formatting step that is important that you have to understand that as a developer.

Olivier Colson: 45:29

So I guess that's also one key thing to have in mind is when you're running your server or deploying your application, adding dependencies, be careful about the dependencies because those are, you know, in the programming world, you use them as black boxes that just produce the output. But here you need to consider what this black box is actually doing because if something is wrongly made into it, well, you might get in trouble depending on how you call it. And so you need to be careful about that as well.

Olivier Dony: 45:57

Yes, you need to be careful. There's actually a field of studio for that called software component and. Analysis where it's there are tools for that, that are going to manage all the dependencies that you have in your apps and big companies, they want to do that for all the software that they install. And at Odoo, of course, we are very careful and very strict about what are the dependencies that we accept to include as part of Odoo because we are also taking all the liabilities with them. When those libraries have vulnerabilities, security problems, well they might be exposed to Odoo as well. Yeah.

Olivier Colson: 46:32

And if something goes wrong you cannot just say, Oh, but it's not us. It's that thing that we were using. It doesn't work like this. Of course you have to take responsibility.

Olivier Dony: 46:39

Because the data is at risk just the same as if you wrote the problematic code yourself. So, effectively, well, I could also talk about SQL injections that are also another kind of vulnerability when you are writing, again, low level code and mixing data into it. But overall, the idea is when you are writing code at the end of the process, instead of just testing that your code is doing what you think it should be doing and basically you are meeting the specification that that you were working on. You also have to ask yourself what could possibly go wrong? You have to take the kind of the opposite side and say, okay, let's imagine I'm an attacker and I want to break on purpose the software. I'm like trying to destroy this bridge, right? So I'm going to pass all the wrong kind of parameters. I'm going to try to abuse the system in all the different manners. And that's where you have a chance to catch those errors early on. And definitely we are trying to make life easy for developers by designing the framework to avoid those mistakes and prevent developers from introducing and falling into those traps. But it's difficult to cover all the cases, and it's like crossing the street. You can have lots of security systems on the cars to try to save life of the people crossing the street, but there is some minimal level of awareness as well that people are working on the street.

Olivier Colson: 48:14

There is no silver bullet something solving everything in all the cases at the end of the day. And within Odoo, how do you work with all these issues? What is done against them and to prevent them in practice?

Olivier Dony: 48:29

So we have many responsibilities. As I was saying, we try to improve the security of the product and also the security of the platform. So, we have people working on reviewing all the code that we are writing for the software and for the platforms. So we work on pull requests and we have the security team reviewing those aspects. Also, we have people working on improving the tools, the tooling for the security. We have, you know, a test suite that is run on every pull request that we produce to verify that the software still is meeting the requirements. But we also have security checks that are being run on every pull request to identify all these red flags.

Olivier Colson: 49:11

Yeah. So it analyzes the code and tries to find patterns of things that look suspicious. And if something looks suspicious, it needs to be checked by your team basically, so that we're sure that we're not introducing something very, very wrong into the code.

Olivier Dony: 49:26

Right. So we try to identify those patterns through static analysis of the code and patterns matching, let's say, access. We know out of experience that access vulnerabilities are introduced when we use certain keywords in JavaScript to perform some certain operations. So we know that if such a keyword occurs, we are going to have a second person have a look, for instance, or to also directly provide recommendations to to the developer. So we have that. And then we also have pen testers. We have people who are actively trying to break the system after the fact, after the code has shipped and is in production. We still need to scan it and try to break it security in order to do it before the bad guys do that. Then of course we have the security program. We run with the community. We have very active researchers in the community. So we are discussing with them all the issues that they think they have found and discussing the possible patches. So that there's a lot of that as well. And then on the platform side, we have also the security of our own database to to cover because.com we do a lot of dogfooding at Odoo. So urdu.com is running on Odoo and that has a lot of exposure as well. So we have to be monitoring the security of that carefully. We have all sorts of protections and different levels of firewalls and in front of that we are designing all the, let's say, policies that we are enforcing with within our company for employees. What is this least privileged roles that we give to our employees depending on their position in the company. So there are lots of aspects to control in the security landscape inside the company. And then, we also have this permanent incident response team that is working on basically our security channels. We have mainly people contacting us at security at odoo.com For all sorts of reasons, for questions, for reviewing pen test assessments that customers and partners run for reporting issues, for asking about situations where they think there is a risk.

Olivier Colson: 51:46

And do we get a lot of reports like this?

Olivier Dony: 51:48

Quite a lot.

Olivier Colson: 51:48

Can you give numbers?

Olivier Dony: 51:50

I'm not sure I should be giving numbers, but I guess the number of email threads is not a very sensitive thing. So we have, I think in the last 12 months we had about 500 email threads on this security channel. So that's several questions a day, a few questions today about all sorts of things, as I was saying, and also about abuse of our platforms. So that's another topic that I forgot to mention. We are also protecting as much as we can the free trials on odoo.com to kind of stop the bad guys. So, there's of course, whenever you are hosting for free websites, you have people who are going to try to abuse that and produce spam or produce phishing campaigns using the free platforms. And of course we are not immune to that and we have lots of heuristics that we develop to analyze the behavior of people doing a free trial to make sure that we can catch early all these people who are using our nice web editor to design a fake bank website to capture the credentials. And it's sad, but it happens every day. Well, it's not a lot I think the numbers is like less than 0.15% of our that of the databases that are created on odoo.com are abusive but that represents a few of them every day and.

Olivier Colson: 53:29

It's good to fight against it first of all because you don't want Odoo to be used this way and second because also for the image of the company it's not good at all.

Olivier Dony: 53:41

Yes. We have to do that. And because we want to also protect the victims, sometimes it's going to be us and we would like the other platforms to do the same. We want as a global community to protect..

Olivier Colson: 53:54

Your responsibility as a software provider to make sure that your software is used in the limits you want it to be used in.

Olivier Dony: 54:02

Right. And that's also something that may not be obvious, but you have to do when you you are hosting a platform like that.

Olivier Colson: 54:10

All right. So we are reaching the end of the episode. I suggest as a conclusion you give us a few elements on the kind of profile Odoo is looking for to take care of security because I guess we are still recruiting for that team as well. So it's time to do a bit of advocacy. What can you what can you tell us? What what do we want?

Olivier Dony: 54:29

So what kind of profile do we have in the security team? In the security world, it's always very siloed and people have very specialized jobs, like watching security events and decide if it's a bad or good event or should I discard it or should I report it to someone else in the chain? And or people are running software scanners and they're scanning and all day they are checking boxes in Excel or to to say, okay this this has checked the market and at Odoo we don't do any such silos. We want people that are generalists and are very curious about the technology and are able to take all these different roles or at one time trying to protect the software, trying to fix the bugs, trying to. And for that you need to understand them, uh, very deeply and at the other times are want to find the bugs and break the system before the attackers can do that. So it's really about being very good developers and being having this state of mind where you are always looking for how to break things and you can take that attacker stance when you you want to destroy the system instead of just making it work, which is the natural tendency for engineers is to make things work and not to break them. Right. So that's the the typical profile.

Olivier Colson: 55:55

Okay. So the word is given. If you feel you match, please apply. Thank you, Olivier, for all your answers. It was super. Were interesting and I guess we learned a lot of things and people can now go and check their deployment and be sure that everything is secured.

Olivier Dony: 56:10

It was a pleasure!

Olivier Colson: 56:12

And that's a wrap for this episode. We've covered a lot of ground today, and I hope you enjoyed discovering more on the ongoing process that software security is. We encourage you to keep learning and implement what you've learned to keep your installations secure and your business protected. If you're interested in more dev and tech content, I recommend you listen to our previous episode where we discussed revamping our JavaScript framework into the fastest in the world with Géry Debongnie. Thank you for tuning in and don't hesitate to follow the show. Leave a review and share it with your colleagues and friends. Until next time, Cheers.