In brief
- AI-based code generation tools are playing a transformative role in automotive software development
- Automakers have a number of benefits and challenges to consider when choosing between flexibility and integration capabilities
- Our comprehensive benchmarking framework for evaluating AI-powered solutions helps automakers navigate the evolving landscape of Generative AI in the automotive industry
Exec summary
Since the introduction of ChatGPT there has been a lot of noise around Generative AI, especially regarding how it can benefit the work of millions of software developers. To understand the real benefits and help organizations make the right choice between the many products appearing on the market, we’ve carried out an assessment on a variety of code generation tools. We conducted automotive-specific coding challenges and found a significant acceleration of development tasks. We also discovered the strengths of individual code generation tools. In this paper, we will express our findings and highlight some challenges we encountered during the assessment and how we overcame them to arrive at our rollout.
Market overview
In the rapidly evolving landscape of automotive, the significance of software development has surpassed its traditional role to become the cornerstone of innovation for software-defined vehicles. With the rise of this paradigm, the automotive industry sits at a pivotal stage where lines of code define our cars’ features, performance and capability to self-repair and update. This paradigm shift not only underlines the relevance of software in enhancing vehicle performance, vehicle safety and user experience, but also emphasizes the need to optimize the entire software development life cycle (SDLC) in an industry known for its cost sensitivity.
Within the SDLC, a significant — if not major — amount of time and resources is invested in the implementation phase. The generative AI tools for software development like GitHub Copilot, Amazon CodeWhisperer, Starcoder, and Tabnine hold the promise of revolutionizing the automotive landscape by streamlining workflows, enhancing productivity and ultimately reducing costs.
However, the adoption of such tools introduces a unique and new set of challenges: Understanding which tool delivers the best performance in terms of coding acceleration (in a complex and very domain-specific development environment such as automotive) while ensuring a smooth integration and superior compatibility with development paradigms, standards, processes, methods and tools. Among these challenges, addressing privacy, functional safety, intellectual property protection and cybersecurity concerns, and maintaining regulatory compliance are paramount.
In this article, we investigate the quickly evolving landscape of generative AI tools for software development, in relation to their application in the automotive industry. We present a comprehensive, rigorous yet efficient, methodology that allows automakers to compare and benchmark generative AI tools for software development in operational, automotive-specific conditions, and to make informed decisions when defining the trajectory of their generative AI transformation journey.
Open source software battling the tech giants
As the automotive industry embraces Generative AI tools for software development, there is a fascinating battle unfolding between open source software and tech giants. Open source AI projects have gained popularity for their collaborative nature and accessibility — these projects allow developers to harness the power of AI without being tied to a specific vendor or technology stack. In the automotive industry, open-source AI initiatives have contributed to advancements in autonomous driving, computer vision and natural language processing.
On the other hand, tech giants like Google, Microsoft and Tesla are developing proprietary generative AI tools tailored to various industry applications, including automotive. These tools often come with the backing of vast resources and expertise, promising cutting-edge solutions. However, they also raise concerns about vendor lock-in, cost and data privacy.
To make an informed decision, automakers should carefully weigh the benefits of open-source solutions which offer flexibility and community-driven innovation, against the potential advantages of tech giants' offerings which may come with enhanced integration capabilities and support.
Change of perspective: Listen to tech leaders
The advent of Generative AI is ushering in a new era in software development within the automotive industry. By harnessing the power of Generative AI, software engineers can now automate and expedite a wide array of tasks, from code generation and debugging to documentation writing. This accelerates the development process and enables developers to focus on more complex, creative aspects of their work. However, the integration of Generative AI necessitates a reevaluation of quality control and ethical considerations, as the code and content generated by AI must meet stringent industry standards and adhere to privacy and security regulations. The role of software developers is shifting toward a more supervisory and creative one, overseeing AI-generated outputs and ensuring they align with the automotive sector's rigorous safety and performance requirements.
For OEMs and Tier 1s, in the automotive industry, AI represents a transformative shift in how they interact with technology and services. This technology offers the potential for highly customized solutions, from AI-driven vehicle fleet management to tailored manufacturing processes. Automakers can benefit from AI's ability to analyze vast amounts of data, providing valuable insights for optimization and cost savings. Additionally, Generative AI can enhance communication and support, automating responses to common queries, and offering more efficient, personalized customer experiences. However, automakers must also consider the implications of data privacy and security when integrating Generative AI into their operations, ensuring compliance with regulations and ethical practices.
New models for AI-enabled organizations
As the automotive industry grapples with the profound implications of Generative AI, forward-thinking companies are reimagining their organizational structures to remain agile and responsive to these technological shifts. In 2022, Zoreza Global started a transformation process to develop a new organizational model called “Guilds and Chapters”. Guilds and Chapters are communities of professionals linked by deep experience and a genuine interest in specific technological topics; groups of individuals with shared skills or expertise who foster knowledge sharing and enhance the quality and consistency of decision-making and technological evolution within the organization.
These groups are communities that transcend traditional team boundaries, encouraging cross-pollination of ideas and skills. This collaborative approach spurs innovation and ensures that the organization can adapt swiftly to technological advancements — such as Generative AI. By fostering these collaborative networks of expertise, the various teams are key players in the decision-making process and allow us to continue to position ourselves as a leader of choice whenever a new technology wave comes along. In this article you can read more about the benefits that an agile-based company structure can bring to companies.
Methodology
Generative AI has revolutionized the landscape of technology, but while the potential for progress is immense, the challenges are equally formidable. To address the uncertainties and hesitations surrounding AI-based tools, our Vehicle Data Insights and Artificial Intelligence Chapter has spearheaded the development of a robust benchmarking framework, tailored to the needs of the automotive industry. This benchmarking framework quickly became a solid tool in the decision-making process for Generative AI and software development. This benchmarking system is designed to rigorously evaluate and assess the efficacy and flexibility of AI-powered solutions. Whether you're a startup looking to make an informed choice or a tech giant seeking optimization of your SDLC for the automotive industry, our process provides invaluable insights tailored to the implementation use cases and technologies that compose your development ecosystem. It's not just a one-size-fits-all solution; it's a flexible, adaptable framework that can be tailored to companies’ specific needs.
To ensure high quality standards for this process, we tested it internally, aiming to identify the most suitable AI-based code generator for our needs, i.e., to cover the main topics that car makers and automotive suppliers have to deal within their product development. First, we conducted a deep market analysis to identify several off-the-shelf candidates to benchmark. In the meantime, thanks to the extensive know-how of our tech leads, we identified a wide range of KPIs and use cases to measure the performance of these products. An indicative selection of KPIs and use cases follows.
KPI categories
- Code performance — how it is impacted by the use of code generators
- Testing — how it is supported by the use of code generators
- Software quality — how it is impacted by the use of code generators
- Coding guidelines — are they respected by code generators?
Use cases
- Write unit tests and cover edge cases
- Get help using unfamiliar libraries
- Adhere to SOLID and DRY principles
- Improve code re-usability
- Implement automatic sanity checks based on Port Connections
- Perform legacy code instrumentation for MISRA / internal guidelines compliance
- Generate ARXML in an AUTOSAR environment
Two independent groups of Zoreza Global engineers created a secure, unbiased, automotive-specific benchmarking environment with an exhaustive set of code challenges and used four AI-based code generators to solve them. The tools used for the assessment were Amazon CodeWhisperer, GitHub Copilot, StarCoder and Tabnine. In parallel, our legal department delved into terms and conditions and defined the policies for using these tools in order to answer questions related to compliance, privacy, security and IP protection.
The work we performed to design and engineer the assets described above allowed us to develop the capabilities needed for successfully accompanying clients and partners in defining and implementing a sustainable Generative AI transition roadmap — a roadmap which considers the specific requirements and constraints of the automotive sector.
These are just some of the features of our benchmarking framework. It has been meticulously devised to rigorously appraise the efficiency and versatility of AI-driven solutions for software development. Within the automotive sector, our framework provides key indicators which align with specific use cases of the development landscape and support the decision-making process at strategic and tactical levels.
Open challenges and next steps
Generative AI presents multifaceted challenges for the automotive industry. Firstly, the adoption of Generative AI in automotive software development raises concerns about intellectual property, as AI-generated content may include code or other assets that constitute intellectual property of third parties. A second point is guaranteeing compliance with ASPICE standards, which defines best practices for software development in the automotive sector: It becomes crucial to ensure the reliability, safety and quality of AI-driven systems. In addition, establishing accountability and standards for AI-generated content is paramount, as companies must ensure that AI aligns with copyright laws and ethical guidelines. Lastly, the integration of Generative AI into the processes related to autonomous vehicles poses challenges related to liability, particularly in the event of accidents: As AI takes on a more prominent role in decision-making, the legal responsibility for AI's actions and decisions becomes a complex matter, raising questions about liability and culpability in case of accidents or errors in autonomous vehicles. Regulators and legal experts are working hard to adapt legal frameworks to address these novel challenges and ensure that the integration of Generative AI in the automotive sector occurs within a comprehensive and well-defined legal framework.
Any AI deployment must follow ethical considerations to at least be compliant with laws such as the EU AI act and at best be inspiring to users. At Zoreza Global, we follow our own internal guidelines for Responsible AI. They are set out here: https://dxc.com/us/en/insights/perspectives/paper/responsible-ai. The first two items which we line out in our Responsible AI guidelines are that “all AI use must be lawful” and “AI must respect data privacy”. As we enter the era of Generative AI, we must apply these guidelines to new technology. Let’s look at the two most contentious discussed legal issues: IP infringement and data leakage. The issue of IP infringement is hotly debated and a major court case which remains unresolved, see https://githubcopilotlitigation.com/. The potential infringement arises if a large language model was trained on proprietary code and as code companion emits that proprietary code to the developer. A major cost risk emerges as unlawful integration of proprietary code into a vehicle might result in the need to stop and recall sold vehicles, rework the code base, and validate and deploy the changes. Data leakage through Generative AI tools has been widely reported, such as the high-profile case of Samsung employees leaking internal source code and meeting notes, reported by The Economist.
Aware of the above challenges and aiming at navigating and anticipating them, Zoreza Global Automotive created a Generative AI Automotive Task Force that looks at this matter from a holistic perspective, combining the know-how brought by AI engineers with the complementary perspective of business leaders and a dedicated legal team. This allowed us to create — among the set of assets that will be revealed in the next blog in this series — the Generative AI Benchmarking framework introduced in this article. As described, this framework is tailored to the needs of the automotive industry and has quickly become a solid partner in decision-making regarding Generative AI and software development. This framework is being adopted by our clients and partners that want to define and implement a sustainable Generative AI transition roadmap, which considers the specific requirements and constraints of the automotive sector. Contact us to have an open discussion on how you could implement and benefit from this framework.
Looking forward
Identifying the right AI tool for accelerating software development in the automotive industry is a highly individualized choice for different actors in the value chain. Various factors influence this decision, such as budget constraints and automotive-specific technical requirements (Functional Safety, ASPICE, Software Quality, to mention some), or — coming to AI-specific aspects — the dynamic, exponentially changing and volatile technology landscape. As the paradigm of software re-usability and hardware abstraction grows within the automotive field, and with it, the lifetime of a portion of code, it is paramount to design, operate and continuously update an AI-based software development toolchain that delivers performance, robustness, scalability, and ensures compliance in terms of IP protection. Ultimately, the choice of a code generator is a strategic decision that should align with an organization's unique priorities, ensuring that it meets their current needs and positions them for future growth and innovation. To find out which code generator would benefit you most, or to have an open discussion on Generative AI for the automotive industry, contact one of our experts.
Giuseppe Calmo
Giuseppe is an accomplished project manager specializing in automotive, robotics, and IT services. Formerly a software engineering manager at Zoreza Global, he worked extensively with leading EMEA car manufacturers in autonomous driving. With a background in research and professorship, Giuseppe holds a European patent in AI for machinery. Currently, as the Western and Southern EMEA leader of Zoreza Global's Vehicle Data and AI Chapter, he actively contributes as a strategy manager, helping solidify Zoreza Global's position as a technological leader and pioneer.
Marco Bottero
Marco is a senior engineering director with a wealth of experience in automotive, travel and transportation, and IT services. He has a distinguished academic background, holding a PhD in engineering. Marco currently leads Zoreza Global’s Engineering team, working on key automotive areas such as autonomous driving, e-mobility and AI, which enable clients to adopt cutting-edge technologies, optimize operational efficiency, and increase the value and impact of their digital products. Outside the professional realm, Marco enjoys scuba diving, sailing, tropical aquariums, and traveling around the world, balancing a dynamic career with personal pursuits.
