đź“–Cooperative Software Development

Amy J. Ko

1. History

  • _

    And that after digital computers, programming wasn’t something that many people did. It was reserved for whoever had access to the mainframe and they wrote their programs on punchcards like the one above.

    (but the image above does not show a punchcard. afaik, the first programming was done on switches and that’s what it looks like on the image)

  • at first, programmers were not solving new unsolved tasks but rather translating existing solutions and make them faster

  • it wasn’t until programmers started solving problems that software engineering was born

  • first truly large software projects were attempted in 1960s

    • first is IBM 360 operating system

  • suddenly, multiple people work on a project, their modules have to cooperate, thus people have to cooperate and software engineering was born

  • first software engineering conference in 1968 (http://homepages.cs.ncl.ac.uk/brian.randell/NATO/nato1968.PDF)

    • asked questions: why projects are late, over budged, and what we can do about it? — many questions and no answers

  • Margaret Hamilton is one of the key people trying to answer these question

    • she established software engineering as engineering. before that, programming was viewed as routine and uninteresting simple work

  • software is becoming so much more:

    • it now should include the question of ethics and responsibility

    • who is responsible that the code is correct?

    • economical roles: software is a major factor now creating and destroying jobs

  • technical aspects of programming has improved steadily—we know a lot about programming languages and testing—however, the human part of software engineering made less progress

2. Organizations

  • q

    even in software organizations, the point of the company is rarely to make software; it’s to provide value.

    • often, the information flowing through the software provides more value than the software itself

    • requirements help organize how software will provide value

  • roles to achieve the value:

    • marketer. looks for opportunities to provide value

    • product manager. decides what value the product will provide

    • designer. decides how software will provide value. envisioning solutions to problems that people have

    • software engineer. implement requirements envisioned by designers

    • sales. takes the implemented product and tries to sell to audiences that marketers choose. provides feedback to marketing, product, and design

    • support. help people to use the product successfully. provide feedback to product, design, and engineering

    • (what are other roles I could think of?)

  • the roles can be merged in one person

  • the division of roles often means that knowledge is not shared between departments, even when that would be helpful

  • be default, engineers are not designers—they do not decide what to build. though they have a great deal of power to influence that

  • other roles:

    • engineering managers. moving information between parts of organization. they are more focused on organization/management rather than actual engineering

    • data scientists. facilitate decision making for other roles.

    • user researchers. help making product decisions

    • ethics and policy specialists. anticipate and prevent harm from engineering and design decisions

  • risks

    • much of organization is focused on mitigating risks

  • reasons why modern open source projects fail https://dl.acm.org/doi/10.1145/3106237.3106246

    1. lost to completion

    2. made obsolete by technology advances

    3. lack of time to volunteer

    4. lack of interest by contributors

    5. outdated technologies

    6. poor maintainability

    7. interpersonal conflicts among developers

    8. legal challenges

    9. acquisitions

  • Cassandra Overney, Jens Meinicke, Christian Kästner, Bogdan Vasilescu (2020). How to not get rich: an empirical study of donations in open source. ACM/IEEE International Conference on Software Engineering.

    • funding open-source projects requires substantial donations from large corporations

    • most projects don’t ask for donations, and those that do receive very little (unless well-established)

    • most of those funds go to paying for basic expenses such as engineering salaries

  • what it means to be a software engineer

    • engineers are not the most important role because their decisions (how to implement) have smaller impact than other roles’ (what to implement, who to sell to)

    • engineers have to work with a lot of people. learning other roles is important to being a good collaborator

    • engineers might have many ideas for product, but they should be in a product role then

    • without engineers, products wouldn’t exist.

3. Communication

  • Good software needs to have conceptual integrity in how it is designed and how it is implemented (Brooks1995)

    • when multiple people are responsible for implementing a single idea, how do they ensure they all build the same idea?

      • effective communication is the answer

  • communication also influences how projects unfold (re: Conway’s law)

  • _

    For example, think back to any course project where you divided the work into chunks and tried to combine them together into a final report at the end. The report and its structure probably mirrored the fact that several distinct people worked on each section of the report, rather than sounding like a single coherent voice. The same things happen in software: if the team writing error messages for a website isn’t talking to the team presenting them, you’re probably going to get a lot of error messages that aren’t so clear, may not fit on screen, and may not be phrased using the terminology of the rest of the site. On the other hand, if those two teams meet regularly to design the error mesages together, communicating their shared knowledge, they might produce a seamless, coherent experience.

    • (this doesn’t seem to follow from Conway’s law directly. rather, it is suggesting that any two modules—if developed by separate team—cannot be integrated seamlessly)

  • communication produces coherent experience for the user

  • technical things are easy to find (who wrote this line of code?), but other things are slow or impossible to retrieve (why this line of code was written?). sometimes, it’s not even clear, who knows the information

    • (working in a large company, this is very true)

  • The Time Famine: Toward a Sociology of Work Time

    • ethnography of work environment found dysfunctional interruptions of most expert members of the team to constantly “fight fires.” Not only this makes experts less productive, but it also makes other parts of organization less likely to find ways of preventing fires in the first place

    • not all interruptions are bad, and they can increase productivity, but they do increase stress (The cost of interrupted work: more speed and stress)

  • a cultural competence: the ability for individuals and organizations to work effectively when their employee’s thoughts, communications, actions, customs, beliefs, values, religions, and social groups vary

  • knowledge sharing tools as a way to reduce time required for communication

    • issue trackers

  • _

    What does all of this mean for you as an individual developer? To put it simply, don’t underestimate the importance of talking. Know who you need to talk to, talk to them frequently, and to the extent that you can, write down what you know both to lessen the demand for talking and mitigate the risk of you not being available, but also to make your knowledge more precise and accessible in the future. It often takes decades for engineers to excel at communication. The very fact that you know why communication is important gives you an critical head start.

4. Productivity

  • it’s hard to define productivity

  • factors that increase productivity

    • better tools

    • interruptions can hinder personal productivity but can increase team productivity

    • skills in architecture design, debugging, testing, programming languages, etc.

    • communication, communication skill

    • learning

      • new engineers tend to hide their lack of knowledge

        New engineers must be even more resourceful, even though their instincts are often to hide their lack of expertise from exactly the people they need help from.

        (cites Novice software developers, all over again)

    • particular features of project’s code, how the projects is managed, environmental and organizational culture

      • these might be the biggest factors

    • personal factors and life

  • barriers to learning programming languages / systems

    • design barrier: abstract computational problem independent of language

    • selection: finding right abstraction or APIs to achieve the design

    • use and coordination: operating and coordinating different parts of language or API

    • comprehension: knowing what can go wrong when using part of a language or API

    • information: limited ability of tools to inspect a program’s behavior at runtime during debugging

  • types of waste (any activity that does not contribute to a product’s value to users or customers) (Software Development Waste | IEEE Conference Publication | IEEE Xplore)

    • building the wrong feature or product

    • mismanaging the backlog: cost of duplicating work, building lower-value features, or delaying necessary bug fixes

    • rework: fixing delivered work

    • unnecessarily complex solutions

    • extraneous cognitive load: e.g., poorly written code, context switching, confusing APIs, or technical debt

    • psychological distress: burdening the team with unhelpful stress from low morale, pace, or interpersonal conflict

    • waiting/multitasking: cost of idle time (often hidden by multi-tasking)

    • knowledge loss: the cost of re-acquiring information that the team once knew

    • ineffective communication: incomplete, incorrect, misleading, inefficient, or absent communication

5. Quality

Software qualities:

  • intrinsic to implementation

    • correctness

    • reliability

    • robustness (recovery from errors)

    • performance

    • portability

    • interoperability

    • security

  • how developers interact with code

    • verifiability

    • maintainability

    • reusability

  • use of software

    • learnability (how easy it is to learn the software)

    • user efficiency

    • accessibility

    • privacy

    • consistency

      • (re: learnability)

    • usability

      • encompasses all of the qualities above

    • bias (discrimination or exclusion by some aspect of the user)

      • inaccessibility is a form of bias

    • usefulness / utility

      • often the most important as it subsumes all of the other qualities

It is impossible to achieve all qualities, and achieving each one takes time, so it’s important to prioritize for each project.

These external notions of quality are not the only qualities that matter. For example, developers often view projects as successful if they offer intrinsically rewarding work. That may sound selfish, but if developers aren’t enjoying their work, they’re probably not going to achieve any of the qualities very well. Moreover, there are many organizational factors that can inhibit developers’ ability to obtain these rewards. Project complexity, internal and external dependencies that are out of a developers control, process barriers, budget limitations, deadlines, poor HR planning, and pressure to ship can all interfere with project success. [emphasis mine]

(cites Why good developers write bad code: an observational case study of the impacts of organizational factors on software quality)

The person responsible for shielding developers from organizational issues is product manager. (I believe it should be project manager.)