Abstraction stacks and multi-paradigm software design

Originally published in Abstraction stacks and multi-paradigm software design, 19 Feb 2004 7:57 PM.

 

Why is important to learn Microsoft Common Intermediate Language (CIL)?

In your role of software technology end-consumer, there is very little, almost non-existent need to.

In your role of software technology producer, a software designer most likely, then the answer is the same as answering: Why is important to a Daytona race pilot to know his engine?

The answer has a relationship with the eagerness of the pilot or the software designer to win!

When my CIL knowledge would be necessary?

From the very first design decision to the last pulse of the crafted bits. That is, if the software artifact is intended to last then it must be designed with that goal in mind.

It seems that most commercial software is designed with other priorities in mind, but not to last. Perhaps, that is a factor behind the reason why, sometimes, end consumers don’t buy more technology.

What is an abstraction stack?

The basic client-server model where a piece of software (the client) asks for something to another software artifact (the server) without caring what the server has to do in order to fulfill the request. This involves the server taking the client role to ask for help to another server, which in turn takes the role of client and…, you got the point.

If you think a little bit about it, an abstraction stack is endless in both directions, server and client, at least philosophically.

The microprocessor (or the running engine like CLR) can be seen as a pragmatic end of current computational abstraction stacks.

Is not the purpose of an abstraction stack supposed to isolate me from knowing all this low-level stuff?

The main purpose of an abstraction stack is to isolate software artifacts, packaging them in layers, subsystems or components of functionality in order to tackle complexity and convey design decisions at higher levels of abstraction.

It is good and desirable for a software artifact to not know the details of its server.

For a software designer, it is neither good nor desirable to not know the details of every layer, subsystem or component in a target abstraction stack.

So, at your next design decision, take in account that you do not have the luxury to not know the details of how your requests are fulfilled. Options are, either the knowledge of the details is in your head or in the head of somebody else in your very same design and programming team.

What is multi-paradigm software design?

Modern software design encompasses a broad set of techniques coming to us from several schools of thought and paradigms. Design skill comes from understanding, articulating, practicing and feed-backing concepts, methods and techniques from all those paradigms.

Multi-paradigm software design is the practice of growing the best possible software in a given context.

“Multi-paradigm design is a domain analysis technique that features concurrent analyses of the application and solution domain. The goal is to find solution domain constructs that most naturally express the structure of the application domain.” As stated by James O. Coplien in “Multi-paradigm design for C++”

If you know CIL, or the details of what ever layer of your abstraction stack, you are at a better position to take proper design decisions.

CIL is another programming language where you have more design options than popular programming languages C# and VB .NET

How this can possibly be related to my current and future job?

Software design is a very singular endeavor at its current state, a combination of art, science, craft and engineering; proper design allows maximum correctness, robustness, modularity and flexibility. Poor design guided by bogus assumptions leads to misery and despair within development teams.

Or in the words of Bertrand Meyer: "The in-the-large aspects of programming rely on the lower-level parts, and you can't get them right unless you get the small things right too"

Obstacles to DevOps

There are several traits of this DevOps inclination that are compelling to me. One is the kind of intellectual promiscuity involved; that is, to combine perspectives of perceivably dissimilar roles in an indiscriminate way. Doing so would enhance the architectural acumen for design decisions by widening the span of attention to even more relevant matters involved is software execution.

There are a number of cultural obstacles to tackle in order to get such acumen in a software provider organization; for example, the obstacles to freely share the hard-won personal technical expertise on the sole premise that doing so is good for the project as a whole, not just good for political exchange of favors among individuals.

Another grievous obstacle is philosophic and scientific illiteracy in the midst of project leaders and team members. Such illiteracy worsen the chasm between the way words are used to map reality in the architectural drawing board, on one hand, and the daily operations room, on the other. A better awareness, for example, of how language can be used in an operational way to build meaning would help decrease the blaming noise and gibberish like ‘our system is down’ or ‘the system does not allow me to give you service’.

Language can be a tool for a clearer thinking, not just a noisy instrument for obscure utterances. A proper use of technical language is important to enhance quality by means of building meaning with sets of operations. That is, of course, there are multiple legitimate ways to use language, but teams using technical language could benefit a lot from the philosophical and historical reflection of the use of language in the sciences. More on that here: Operationalism

Why a Reflective Developer Program?

I have written down the rationale behind the Reflective Developer Program in the following pages, ending with a brief introduction to the program:

1. The state of software development in some trenches.

2. Agile impressions.

3. The Law of Raspberry Jam.

4. Learning from other communities of inquiry.

5. The Reflective Developer Program - what is it about?

4. Learning from other communities of inquiry

There are a number of works on epistemology —as philosophy of science— and on general theory of knowledge —or gnoseology, as known in some Latin-derived cultures— that have been very significant on my research. For example, Ernst Cassirer’s works study knowledge as a philosophical problem and its implications on the development of human culture:

- The Problem of Knowledge.

- The Philosophy of Symbolic Forms: Vol. 3: The Phenomenology of Knowledge.

- An Essay on Man: An Introduction to a Philosophy of Human Culture.

The philosophical perspectives predominate over strictly scientific perspectives in Cassirer’s works. I have found a more balanced treatment between philosophy and science in Mario Bunge’s two volumes on epistemology:

- Philosophy of Science: From Problem to Theory.

- Philosophy of Science: From Explanation to Justification.

Introductory works more on the side of strictly scientific perspectives that I have found very helpful are:

- What Is This Thing Called Science? by Alan F. Chalmers.

- Scientific Method in Practice by Hugh G. Gauch Jr.

- Theory and Reality: An Introduction to the Philosophy of Science by Peter Godfrey-Smith.

Then, there is this contemporary scientist, Rupert Sheldrake, who —as Paul Feyerabend in his own time— presents quite intriguing, or even interesting, arguments that shake popular understanding of scientific thought in his work:

- Science Set Free: 10 Paths to New Discovery, USA Edition.

- The Science Delusion, UK Edition.

Among the first works that indirectly pushed me to all the mentioned readings was this:

- Testing Computer Software, 2nd Edition by Cem Kaner, Jack Falk, Hung Q. Nguyen.

All this, and the following works by Donald A. Schön, has led me to pursue the development of what I have called a Reflective Developer Program, as a contribution to better professionalism in software development. A reflective developer is someone who tries to realize how thin her received piece of jam really is, and tries to make it thicker.

- The Reflective Practitioner: How Professionals Think In Action.

- Educating the Reflective Practitioner: Toward a New Design for Teaching and Learning in the Professions.

The Reflective Developer Program is intended to be a personal research journey to better understanding of professionalism in computing. Given the philosophical problem of knowledge and The Law of Raspberry Jam, I think that teaching and learning in our profession need to be rethought. Therefore, after studying the following works, I have found that dialog and discussion among practitioners is the appropriate thing to do.

- Turning Learning Right Side Up: Putting Education Back on Track by Russell L. Ackoff, Daniel Greenberg.

- Up and Out: Using Critical and Creative Thinking Skills to Enhance Learning by Andrew P. Johnson.

- Discussion as a Way of Teaching: Tools and Techniques for Democratic Classrooms by Stephen D. Brookfield, Stephen Preskill.

- Developing Critical Thinkers: Challenging Adults to Explore Alternative Ways of Thinking and Acting by Stephen D. Brookfield.

3. The Law of Raspberry Jam

«The Law of Raspberry Jam: The wider you spread it, the thinner it gets.» —Gerald M. Weinberg

As Gerald M. Weinberg said in his work The Secrets of Consulting about The Law of Raspberry Jam: «Another way of expressing the law is this: Influence or affluence; take your choice.» That is, the wider the audience, the more money you can make. Based on the observed effects so far, maybe we could know which has been the choice of the authors of ideas like the agile methods. I am glad they chose to spread their ideas; otherwise, maybe, I could not hear anything about such wonderful ones.

After some years studying about the philosophical problem of knowledge, I think that the choice of influence —as in The Law of Raspberry Jam, alternatively expressed— is exclusively a personal one. That is, I have received my part of jam, as thinner as it could be; now it is my choice to make it thick by means of self-influence: by personally researching the subject matter in order to get the more jam as I could possibly grasp.

For example, in my own research efforts, one of my null hypothesis is that the actual properties of the outcome of a non-trivial software development effort is an epistemic problem; that is, the actual properties of the outcome cannot be known until such outcome is actually experienced by its ultimate customers or end users; most previously conceived rationalizations must be doubted and put to rigorous empirical examination. Another null hypothesis is that the previous one will continually be true indefinitely because of the problem-solution coevolution model of design, where the sole presence of a solution to a complex problem —or a subset of the solution— redefines such a problem.

A trait of my provisional conclusions is that the very activity of software development for non-trivial projects is an activity of inquiry, of an experimental nature. This is true regardless past, current and future buzzwords because most of them, e.g., agile, are not intended to bring new knowledge but just new wrappings for already conceived and published ideas of the past. Part of the support for such conclusions can be traced in the works by early authors in our field, e.g., Gerald M. Weinberg, Frederick P. Brooks, Jr.

The practice of discussing these topics in communities of inquiry, and learning from the feedback, is another way to self-influence.

2. Agile impressions

Why are things as they are? How might things be different? Some years ago I started a personal inquiry guided by questions like these. Just recently I read the Agile Impressions text by Gerald M. Weinberg. I have read many of Weinberg’s books, and I have read that some Agile Manifesto’s authors have learned a lot from those books. That is why I am especially interested to read what Gerald M. Weinberg has to say about the Agile Manifesto’s mindset.

The section Agile and the Definition of Quality, as properly stated at the beginning, is adapted from section 1.1 A Tale of Software Quality, in Quality Software Management. Volume 1: Systems Thinking. I noticed that Weinberg equates the actions of a quality manager —as of 1991— with the actions of an Agile team —as of 2013—.

1991: «That’s why one of the most important actions of a quality manager is to bring such decisions into consciousness, if not always into public awareness.»

2013: «That’s why one of the most important actions of an Agile team is bringing such decisions into consciousness, if not always into public awareness.»

That is pretty much consistent, historically, with the quotation attributed to Weinberg in the article about the history of iterative and incremental development, by Craig Larman and Vic Basili: «...where the technique used was, as far as I can tell, indistinguishable from XP [eXtreme Programming].»

In other words, this is additional evidence for many possible conclusions; for example, that iterative and incremental development (IID) was not born with object-oriented methods in the 90’s, or that IID is not recent and comparatively unproven. Also, this is evidence of how true David Parnas statement is about where the next big thing in software engineering lies, in the horizon or in the past, in the history of our profession.

1. The state of software development in some trenches

The industry of software development has a history of its own ideas like structured analysis, design and programming, or object-oriented, or functional programming, agile methods, software architecture, and so on. In order to learn about some of those ideas, over many years, I have been involved with communities that form around those ideas, learning from authors and practitioners, and from my own application of those ideas to improve the quality of my own software design and programming practice. Along the way, I have tried to discuss with others about all that good stuff of software development as a cooperative game.

As years come and go, I have found teams and individuals that expose themselves to be influenced by ideas in the industry around, and the degree of such influence varies widely. There are still lots of programmers and managers who simply do not care enough to not just repeat the buzzwords but to behave like if they properly understood the concepts. Comes to mind what have happened with other intellectual pursuits in history. For example, very few people around me, even with several academic computing degrees, are able to articulate, in simple terms, the most general justified true beliefs from quantum theory, much less its relationship with the current computing industry. Whereas those justified true beliefs were published almost a century ago!