No silver bullets (part 3...the last one i promise)
Wrapping up exploring paradigms for end-user programming interactions
This is the final post for chapter 4 of Bonnie A. Nardi's "A Small Matter of Programming". In the previous two posts, we covered four of the techniques Nardi mentions in this chapter, so make sure to go check those out if you haven't yet. We'll talk about the last technique here, and some overall takeaways from this chapter and how it's relevant to end-user programming design today. Let's get into it!
Interaction Techniques for End User Application Development (continued)
Automatic Programming by Informal Program Specification
This, like the other technique titles, is just a longer way to say a pretty simple concept: automatically creating a program. It's a sorta futuristic concept, shifting all the burden of programming onto the computer, minimizing the effort for the end-user.
The model of the interaction is less that of a programmer with a compiler and more that of a client with a systems analyst (p.78)
Of course, this technique is far more easily said than done. If we want to evaluate it, we have to actually understand what goes into a program generating system. There's three components that Nardi wants to look at:
How you actually define the requirements of the program you want
The way you communicate these requirements into the system
How you determine if the output program actually works
Let's look at each of these pieces.
1. Specifying requirements
Folks who are non-programmers understand software from its behavior and interface. For example, if you casually asked someone how Spotify works, they'd say something along the lines of "you click or tap on a song, and it plays the song". You wouldn't hear "you fire the event that requests the song from Spotify's databases, and the application streams the audio data to your computer's speakers", unless you were talking to an engineer who was being unnecessarily pedantic.
Nardi's suggestion is that with a method like this, you would want an interface that lets you describe your program through its behavior. One way would be to demonstrate it (programming by example)—the other way would be describing the program in a form that is "less prescribed than traditional programming languages". So we talked about the problems of the first approach, and this second approach...wait, that sounds a lot like programming still!
Simply using a specification language as many existing automatic programming systems do,...is, in reality, doing little more than replacing one programming language by another as for as the end user is concerned. (p.79)
Nardi references two studies which essentially find that people are not too great at understanding how to define requirements that actually aligns with what the system is capable of. If given no constraints, then behavior gets defined in ways the system can't support. If given real constraints, people become confused on how they can put things together to assemble a program. Identifying what the user can have in their program is just as important as figuring out what they want in their program.
Despite the fact that these referenced studies were done in the 1990, I imagine we'd find a similar result today if we ran those same studies. Folks just generally aren't good at grasping this "top down" approach of specifying program requirements within the constraints of most systems.
That being said, I am aware that folks have been working on building program specification systems that are far more flexible than the ones that existed 30 years ago, so maybe we'll see something that works in this realm soon!
2. The Form of Communication
We've looked at the macro level of communication—this (extremely short) second section addresses the medium of communication itself. We already looked at the issues of natural language as an interface in "Talking to robots". Nardi suggests that the proper medium exists somewhere halfway between human and computers, giving suggestions like forms, task-specific languages (like spreadsheet formulas) and drawing tools.
Personally, I'm finding that searching for this "top down" generation approach isn't the way to go, and that the proper "medium" is actually a set of understandable building blocks that make sense to both people and computers and can be built with from the ground up. For example, Notion has blocks that can be dragged around and pages that hold blocks. Spreadsheet tools have cells and formulas that connect them. Figma has shapes and frames.
Once these building blocks "primitives" exist, end-users can then dream of how they can be put together to make what they're looking for. Yes, there's some learning required, but any medium of communication will require some level of learning. With primitives though, you give the end-user the ability to imagine how far their programs can go.
3. Evaluation and the Identification of Requirements
Like we mentioned earlier, people make errors. Even with a "perfect" specification and program generation system, it's incredibly unlikely that end-users will generate a perfect program the first time. So when this faulty program is generated, how will end-users know what to fix from it?
In software, this process is called "debugging". However, with code this usually requires an understanding of the abstraction layer "deeper" than the one you programmed in. This isn't an option for end-user programming, so how else could we do it?
Nardi proposes a method of generating a "candidate program". This program would act as a concrete reference that a dialogue takes the end-user through, prompting the end-user to "iron out any misunderstandings and achieve a program closer to the users' requirements".
Ideally, users would be able to incrementally change their program with this dialogue, fixing things as they come across errors. The method for fixing the program should be roughly similar to the process of setting the program up, as both patterns will share many similar elements. Another benefit from having a concrete example is that this example can act as a prototype, letting users carry out specific tasks to identify where things go wrong.
Though I definitely agree with the usefulness of having a concrete program, there's still the challenge of surfacing what exactly is "wrong" with a program. How do you define what's wrong? If you wanted the second name in a list of names, but instead programmed for the first, how would the dialogue understand that you actually meant the second name?
Furthermore, how would a dialogue take you through the specific steps that could be wrong in a program in the first place, without getting into code? After looking at these techniques, this automatic program generation approach doesn't seem to be what we're looking for.
The utopian picture is that of a system so advanced that users have merely to tell it what they want, and it will build it for them. If, however, users do not know what they want, then such a system might prove intractable. If users cannot give a description of the desired program, nor describe their requirements, nor effectively evaluate any programs generated by the system, then the dialogue envisaged will fail to provide the appropriate support. (p.83)
TL;DR for this technique: Automatic program generation is just three smaller programming problems in a trenchcoat, and these smaller problems are not easy to solve
WE'RE FINALLY DONE! Let's look at why any of this is relevant for 2021
So we've finally gotten to the end of the five techniques Nardi breaks down in this chapter. In each section, I talked a little bit about the challenges facing each one, and even tried and introduced modern developments in each section if they're relevant.
But like...there has to be more techniques, right? Stuff that Nardi didn't look at, or stuff that wasn't around yet? Well...yes, of course! Since the 30-ish years since this book's publications, quite a lot of developments have happened. Folks have gravitated towards certain concepts and paradigms, and technology has allowed for high-computation features like Search to become far more powerful than what was available in the 1990s.
One particular interaction technique I've seen rise in popularity quite tremendously is "direct manipulation". You can read a good technical breakdown of direct manipulation here, but the general idea is that with incredibly fast feedback on input, users can grasp the tools at their disposal far quicker than traditional paradigms. The most obvious direct manipulation action is dragging an "object" on touch screens, like swiping a card right, or cropping an image.
In a desktop setting, direct manipulation is pretty popular in editing and design tools. The immediate feedback on actions lets the user visually iterate incredibly fast.
They're starting to appear in other types of tools as well though! Notion is a great example, with blocks being movable objects that contain data.
While this technique isn't specific to end-user programming, it certainly has been proven by many of these tools (Notion, Figma, Webflow, even spreadsheets) to be very useful. The level of tangibility you get from the immediate feedback not only helps folks understand the results of their inputs—it also builds confidence in the tools that are available to them.
As for tradeoffs, direct manipulation does run into issues with accessibility, and puts a layer of interaction "opinion" onto your model, so this technique should be used very thoughtfully!
I hope that as more and more folks are entering the industry as well, we can start to converge on commonly accepted paradigms and interactions. Nardi suggests that there could be utility on building a "taxonomy" of interaction techniques, so we don't have to start from scratch each time.
Luckily, this is an area that has had a significant amount of development. Organizations like the Nielsen-Norman Group (where the direct manipulation link above goes to) have been pretty good at documenting and breaking down specific techniques. It's definitely a little dated, but still very useful!
Hopefully as end-user programming gains more traction, designers in the field can identify higher-level, more "abstract" system design paradigms, as well as their tradeoffs and constraints.
This chapter ended up taking quite a long time to write through. There was definitely a lot of content to break down, and hopefully I did a good job of simplifying things for future me.
In the next post, we're going to start digging into specific visual paradigms. We look at the benefits mentioned in the first technique, "Visual Programming", especially when text is introduced. We'll look at "visual formalisms" and why they have proven to be such a great concept for system design. It'll definitely be less abstract than these posts, and hopefully less lengthy 😅 Catch you then!