Industrial Engineering Knowledge Center: Productivity Science of Software Development Process, Facilities and Systems

Thursday, December 7, 2023

Productivity Science of Software Development Process, Facilities and Systems

Productivity Science is the first principle of Industrial Engineering Principles

IISE conference paper: Principles of Industrial Engineering. 2017 IISE Annual Conference Proceedings; Norcross (2017): 890-895.

https://www.proquest.com/docview/1951119980

http://nraoiekc.blogspot.com/2012/03/industrial-engineering-principles.html

https://www.linkedin.com/pulse/principles-lean-industrial-engineering-narayana-rao-kvss/

Productivity Science of Software Development - What affects software productivity and how do we improve it?

1994

Software Development Productivity Drivers

The attributes of a software project that facilitate high productivity include:

Software Development Environment Attributes:

Fast turnaround development activities and high-bandwidth processing throughput (may require more powerful or greater capacity computing resources)

Substantial computing infrastructure (abundant computing resources and easy-to-access support system specialists)

Contemporary software engineering tools and techniques (use of design and code development aids such as rapid prototyping tools, application generators, domain-specific (reusable) software components, etc., used to produce incrementally development and released software products.)

System development aids for coordinating LSS projects (configuration management systems, software testing tools, documentation management systems, electronic mail, networked development systems, etc.)

Programming languages with constructs closely matched to application domain concepts (e.g., object-oriented languages, spreadsheet languages)

Process-centered software development environments that can accomodate multiple shifting patterns of small group work structures

Software System Product Attributes:

Develop small-to-medium complexity systems (complexity indicated by size of source code delivered, functional coupling, and functional cohesion)

Reuse software that supports the information processing tasks required by the application

No real-time or distributed systems software to be developed

Minimal constraints for validation of data processing accuracy, security, and ease of alteration

Stable system requirements and specifications

Short development schedules to minimize chance for project circumstances to change

Project Staff Attributes:

Small, well-organized project teams. Large teams should be organized into small groups of 3-7 experienced developers, comfortable working with each other

Experienced software development staff (better if they are already familiar with application system domain, or similar system development projects)

Software developers and managers who collect and evaluate their own software production data and are rewarded or acknowledged for producing high data value software

A variety of teamwork structures and the patterns of shifts between them during task performance.

Software Development Cost Drivers

The factors that drive software costs up should be apparent from this list of productivity drivers. Software cost drivers are the opposite of productivity drivers. For example, software without real-time performance should be produced more productively or at lower cost than comparable software with real-time performance requirements.

Also, it should be clear from this list that it is not always possible or desirable to achieve software productivity enhancements through all of the project characteristics listed above. For example, if the purpose of a project is to convert the operation of a real-time communications system from one computer and operating system to another computer-operating system combination, then only some of the characteristics may apply favorably, while others are inverted, occurring only as inhibitors. In this example, conversion suggests a high potential for substantial reuse of the existing source code. However, if the new code added for the conversion affects the system's real-time performance, or is spread throughout the system, then productivity should decrease and the cost increase. Similarly, if the conversion is performed by a well-organized team of developers already experienced with the system, then they should complete the conversion more productively than if a larger team of newly hired programmers is assigned the same responsibility.

UNDERSTANDING SOFTWARE PRODUCTIVITY

WALT SCACCHI

Information and Operations Management Department

School of Business Administration, University of Southern California, Los Angeles, CA 90089-1421, USA

(Appears in Advances in Software Engineering and Knowledge Engineering, D. Hurley (ed.), Volume 4, pp. 37-70, (1995). December 1994

https://ics.uci.edu/~wscacchi/Papers/Vintage/Software_Productivity.html

2022

What Improves Developer Productivity at Google?

Lan Cheng et al, Google, United States

https://dl.acm.org/doi/pdf/10.1145/3540250.3558940

Independent Variables Used in the Model

4.3.1 Code Quality & Technical Debt. The first category of potential drivers of productivity are those relating to code quality and technical debt.

In an experiment, Schankin and colleagues found that participants found errors 14% faster when descriptive identifier names were used. Studying small industrial program written in the 1980s, Gill and Kemerer found that code complexity correlates with software maintenance productivity. Based on interviews and surveys with professional software developers, Besker and colleagues found that technical debt correlates negatively with developer productivity.

We measured code quality and technical debt with 5 subjective factors from our survey:

• Code Quality Satisfaction (sat. with project code quality, sat. with dependency code quality)

• Code Technical Debt (project tech debt)

• Dependency Technical Debt (dependency tech debt)

• Technical Debt Hindrance (tech debt hindrance)

4.3.2 Infrastructure Tools & Support. The next category of potential drivers of productivity are issues relating to tools and infrastructure. Prior work showed that using the best tools and practices was the strongest correlate of individual productivity at Google, though not a significant correlate at two other companies. Storey and colleagues also found that Microsoft developers’ processes and tools correlated with individual productivity.

This category had 6 objective and 12 subjective measures:

• Tools, infrastructure and service satisfaction (sat. with infra & tools)

• Tools and infrastructure choice (choices of infra & tools)

• Tools and infrastructure innovativeness (innovation of infra & tools)

• Tools and infrastructure ease (ease of infra & tools)

• Tools and infrastructure frustration (frustration of infra & tools)

• Developer stack change (change of tool stack)

• Internal documentation support (doc. support)

• Internal documentation hindrance (doc. hindrance)

• Build & test cycle hindrance (build & test cycle hindrance)

• Build latency satisfaction (sat. with build latency)

• 50th and 90th percentile of build duration (p50 build time, p90 build time)

• % of long builds per week (% of long builds)

• 50th and 90th percentile of test duration (p50 test time, p90 test time)

• % of long tests per week (% of long tests)

• Learning hindrance (learning hindrance)

• Migration hindrance (migration hindrance)

4.3.3 Team Communication. The next category of drivers of productivity are issues relating to team communication. In a survey of knowledge workers, Hernaus and Mikulić found that social job characteristics (e.g. group cooperation) correlated with contextual job performance. More specifically, in software engineering, Chatzoglou and Macaulay interviewed software developers, finding that most believed that communication among team members was very important to project success. Studying communication networks quantitatively, Kidane and Gloor found that in the Eclipse project, a higher frequency of communication between developer correlated positively with performance and creativity.

To measure team communication in our study, we examined 9 objective measures and 1 subjective measure:

• 50th and 90th percentile of rounds of code review (p50 code review rounds, p90 code review rounds)

• 50th and 90th percentile of total wait time of code review (p50 code review wait time, p90 code review wait time)

• 50th and 90th percentile of code reviewers’ organizational distances from author (p50 review org distance, p90 review org distance)

• 50th and 90th percentile of code reviewers’ physical distances from author (p50 review physical distance, p90 review physical distance)

• Physical distance from direct manager (distance from manager)

• Code review hindrance (slow code review)

4.3.4 Goals and Priorities. Prior research suggests that changing goals and priorities correlate with software engineering outcomes.

Surveying 365 software developers, The Standish Group found that changing requirements was a common stated reason for project failure. Meyer and colleagues found that one of the top 5 most commonly mentioned reasons for a productive workday was having clear goals and requirements.

We measure this category with 1 subjective measure:

• Priority shift (priority shift)

4.3.5 Interruptions. Meyer and colleagues found that two of the top five most commonly mentioned reasons for a productive workday by 379 software developers was having no meetings and few interruptions. Similarly, a prior survey of Google engineers showed that lack of interruptions and efficient meetings correlated with personal productivity, as did use of personal judgment.

We measure this category with 3 objective measures:

• 50th and 90th percentile of total time spent on incoming meetings per week (p50 meeting time, p90 meeting time)

• Total time spent on any meetings per week (total meeting time)

4.3.6 Organizational and Process Factors. Finally, outside of software engineering, organizational and process factors correlate with a variety of work outcomes. For example, according to healthcare industry managers, reorganizations can result in workers’ sense of powerlessness, inadequacy, and burnout. Although not well-studied in software engineering, based on personal experience, DeMarco and Lister [11] and Armour [2] point to bureaucracy and reorganizations as leading to poor software engineering outcomes.

This category had 2 subjective and 3 objective measures:

• Process hindrance (complicated processes)

• Organizational hindrance (team & org change)

• Number of times when engineers’ direct manager changes but colleagues do not change (reorg direct manager change)

• Number of times when both an engineer’s direct manager and colleagues change simultaneously (non-reorg direct manager change)

• Number of different primary teams the engineer has (primary team change)

Metrics’ relationship with self-rated productivity. - Significant Variables

Metric Effect size p-value

Code Quality & Technical Debt

sat. with project code quality 0.105 <0.001

project tech debt 0.078 <0.001

dependency tech debt 0.042 0.012

Infrastructure Tools & Support

sat. with infra & tools 0.113 <0.001

choices of infra & tools 0.020 0.083

innovation of infra & tools 0.106 <0.001

change of tool stack 0.019 0.098

build and test cycle hindrance 0.029 0.064

p90 build time -0.024 0.019

learning hindrance 0.038 0.006

Team Communication

p50 code review rounds 0.007 0.081

p90 code review rounds -0.014 0.058

slow code review 0.051 0.004

Goals & Priorities

priority shift 0.077 <0.001

Interruptions

Organizational Change and Process

complicated processes 0.027 0.067

team and org change 0.032 0.023

reorg direct manager changes -0.002 0.086

primary team change 0.014 0.086

Factors Causally Linked to Productivity

• For code quality, we found that perceived productivity is causally related to satisfaction with project code quality but not causally related to satisfaction with code quality in dependencies. For technical debt, we found perceived productivity is causally related to perceived technical debt both within projects and in their dependencies.

• For infrastructure, several factors closely related to internal infrastructure and tools showed a significant causal relationship with perceived productivity:

ś Engineers who reported their tools and infrastructure were not innovative were more likely to report lower productivity.

ś Engineers who reported the number of choices were either too few or too many were likely to report lower productivity. We further tested whether one of the two (łtoo fewž or łtoo manyž) matters but not the other, by replacing this variable with two binary variables, one representing the case of łtoo fewž choices and the other representing the case of łtoo manyž choices. The results suggest that both

cases are causally related to perceived productivity.

ś Engineers who reported that the pace of changes in the developer tool stack was too fast or too slow were likely to report lower productivity. Similarly, we tested the two cases, łtoo fastž or łtoo slowž, separately by replacing this variable with two binary variables, one representing the case of łtoo fastž and the other representing the case of łtoo slowž. Results suggested both cases matter for

perceived productivity.

ś Engineers who were hindered by learning a new platform, framework, technology, or infrastructure were likely to report lower productivity.

ś Engineers who had longer build times or reported being hindered by their build & test cycle were more likely to report lower productivity.

• For team communication, a metric related to code review was significantly causally related with perceived productivity. Engineers who had more rounds of reviews per code review or reported being hindered by slow code review processes were likely to report lower productivity.

• For goals and priorities, engineers hindered by shifting project priorities were likely to report lower productivity.

• Organizational factors were linked to perceived productivity:

ś Engineers who had more changes of direct managers were more likely to report lower productivity.

ś Engineers who reported being hindered for team and organizational reasons, or by complicated processes were more likely to report lower productivity.

Bibliography

https://thenewstack.io/platform-teams-adopt-these-7-developer-productivity-drivers/

https://www.mckinsey.com/~/media/McKinsey/McKinsey%20Solutions/Numetrics/Resources/Insights%20on%20%20Medical%20devices%20Numetrics.pdf

https://www.mckinsey.com/~/media/McKinsey/McKinsey%20Solutions/Numetrics/Resources/Insights%20on%20Networking%20Numetrics.pdf

https://www.researchgate.net/publication/221494845_Productivity_trends_in_incremental_and_iterative_software_development

https://link.springer.com/chapter/10.1007/978-1-4842-4221-6_7

https://uweb.engr.arizona.edu/~ece473/readings/9-Improving%20Speed%20and%20PRoductivity.pdf

https://ics.uci.edu/~wscacchi/Presentations/Process/Software-Productivity.ppt

https://www.logilica.com/blog/what-is-the-space-framework-for-developer-productivity

2022

What Improves Developer Productivity at Google?

Code Quality