The " DEVOPS·2023China Annual Report " organized and launched by DevOps+ Lab at Nanjing University collected the latest development status and trends of software R&D effectiveness and DevOps (Integration of Development and Operations) from experts and practitioners in the domestic software and IT industries through a survey questionnaire since August 2023. After two months of questionnaire collection and subsequent statistical analysis of the questionnaire data, the Lab officially released the "Software R&D Effectiveness · 2023 China Annual Survey Report" on December 30, 2023《DEVOPS·2023China Annual Report》（hereinafter referred to as《Annual Report》，you can download the full version of the report by scanning the QR code.）。

Since the summer of 2016, DevOps+ Lab at Nanjing University has been the first in China to conduct the "DevOps· China Annual Survey" and has continuously released the "Annual Survey Report" for six consecutive years, generating widespread industry impact. In order to more fully reflect the latest development trends of the software industry, the 2023 annual survey activity was further upgraded to the "DevOps· China Annual Survey". The upgraded "Annual Survey Report" continues to adhere to a vendor-neutral perspective of non-profit, comprehensively reflects and comprehensively analyzes the latest development status of software R&D effectiveness and DevOps in China, presenting horizontal international and domestic reference comparisons as well as multi-dimensional perspectives of vertical industry trend evolution. The continuous release of the "Annual Survey Report" has attracted widespread attention and positive feedback in the industry and academia, and provided important data references for the development of relevant national standards. On December 1, 2023, the "Systems and Software Engineering Development and Operation Integration Capability Maturity Model" national standard (standard number GB/T 42560-2023) was officially implemented nationwide, which was jointly drafted and formulated by the National Information Technology Standardization Committee as the lead organization, with Nanjing University's DevOps+ Lab as the core team and 34 other organizations as participants.

With the rapid popularization of DevOps, as well as the rise of related R&D concepts such as DevSecOps, AIOps, and MLOps, R&D effectiveness has become a hot field of software engineering that is highly concerned by both Internet companies and traditional software enterprises in today's "pan-DevOps era". The 2023 annual survey activity continues the backdrop of DevOps, highlights the theme of software R&D effectiveness as the perspective for questionnaire design and analysis, and conducts in-depth analysis of annual hot topics such as software security and large language models. The survey results show that more than half of the surveyed organizations have begun to apply DevSecOps or integrate large language model (LLM) tools into their practices to optimize R&D effectiveness.

The survey questionnaire was distributed within 60 days of the start of the survey, and a large number of responses were received. The "Annual Survey Report" conducted statistical analysis on the valid questionnaire data. The report comprehensively analyzed the current status of R&D effectiveness from two aspects: metrics and practices. It used eight standard metrics to classify the effectiveness of the surveyed teams, including deployment frequency, delivery cycle, average time to repair, failure rate of changes, effective meeting time, code rework rate, customer satisfaction, and average time between failures. Through the analysis of different effectiveness-classified teams, the report identified constraints and best practices for R&D effectiveness. Additionally, the report presented differences in software security practices among teams of different effectiveness levels from different perspectives. This year's large language models have undoubtedly become a hot topic in software engineering. The "Annual Survey Report" also conducted quantitative analysis and discussion on the impact of LLM on software development models, as well as the risks and challenges it brings.

In addition to Nanjing University's DevOps+ Lab as the initiating organization, the 2023 China Annual Survey activity has also received strong support from multiple authoritative institutions and well-known technology media organizations in China. The event declined the participation and commercial sponsorship of vendors.

Sponsor：

Co-operation：

Download: