Week 15: Final Project Update

This will be my final update for the Studio II project. I feel a complex blend of emotions as I write this. I am relieved to be done. I am also sad to know that my time with this team has come to an end. I consider myself incredibly lucky to have spent so much time working with some truly amazing designers. I don’t know if I will ever experience anything like this again, but I hope so.

Remote collaboration has few perks, and I was lucky to be working with folks who helped to make this experience so much fun

The work we have done this week feels different for many reasons. We had to prepare something for a large and diverse audience, not all of which knew or were familiar with the context of our work. Additionally, we also needed to use this time to tie up remaining loose ends—we needed to reach an end state where our process could feel somewhat conclusive.

Our efforts were just as collaborative as ever, as we divided up the labor of our remaining tasks. I was incredibly reassuring to know my team members strengths and capabilities. Knowing who was working on a particular task was reassuring. For my part, I was busy scrubbing through a timeline in After Effects, rapidly assembling visual representations and edited footage to make a convincing newscast from the future. Considering the constraints of remote collaboration, I’m very pleased with the final product.

I have continued to ruminate about over this notion that the future is something we cannot predict, but rather something we build through imperfect knowledge. I question the power our team has to influence this process, not because I lack the confidence in our shared abilities —as I said earlier and often, I’ve been working with an amazing team— but more of a concern around consequences of inspiration. Our process was far from perfect. The vagaries of a pandemic distorted every effort. The educators we sought to connect with were terribly busy. Our own team suffered from fatigue and sleeplessness as we juggled future careers and other academic expectations. The complexity of this topic is well beyond the scope of fifteen weeks of diligent inquiry.

I cannot speak for the entire team, but I know that for me personally our exploratory research was the most intimidating phase. It was immediately clear that we were engaging in a very difficult problem. Education intersects with so many other areas of study. It is a problem of policy, culture, funding, methodologies, and it is weighed down by a history of systemic inequality and racism. Generative research methods were the biggest surprise. I was astonished by what could be gleaned through a participatory process. Including educators in the generation of concepts was exciting, and I wish we had more time to engage in this work.

Our final concepts are a reflection of many perspectives and early prototypes generated by K-12 educators

Every phase also felt too short. We needed to move on before we could fully digest what we were learning. Nevertheless, I stand behind the work we have done, because I know it represents the best we had to offer. I’ve known that design is a messy process long before my time at CMU, but I now have a much clearer sense of what it means to engage with that mess and to assemble something coherent. This work is not easy, and it is never, ever truly complete. The deadlines for a design project function like the layers of silt in a fossil record. The strata of every layer represents a progression with no clear ending or beginning. We can always dig deeper.

I hope these artifacts will inspire others as they have inspired us.

Our team has assembled a project homepage. There you will find more comprehensive information about this work, the final outcome and documentation. Check it out!

Week 11: qualitative evaluation of concepts

Our online survey is now underway, and while this virtual format isn’t exactly like so-called “speed dating,” we are hoping that it will be able to serve a similar purpose for our research. Creating a meaningful online experience for our participants was a tall order, especially with such tight constrains. There are many risks when created a fully automated and hands-off system. Not being there to clarify or to address questions or concerns in realtime was something we needed to accept as a trade-off. In exchange, we have a dozen unique participants ranging from 2 years to 27 years of experience, and from various districts around the country.

So far, the majority of responses have been from an online community of English teachers, so our data is skewed toward this perspective. On the plus side, English teachers provide excellent written responses. To avoid the pitfalls of statistics and quantitative analysis, we designed an online survey with open text fields, and we framed our questions around hypothetical scenarios. This would provide us with reflection and insights into how teachers imagine these concepts for themselves, and what perceived deficiencies come up for them in thinking about these systems in action.

Screenshot of survey responses, exported into a CSV file

The last 24 hours in particular have been very exciting, as we finally gained access to online educator communities. This process has been slower than wanted, but we first needed to fully develop our survey before we could deploy it. This process in and of itself was a design challenge. 

Last weekend, we decided to use the Tripetto platform. This gave us the same logic capabilities as TypeForm, but without any additional costs. It became clear almost immediately that we would need to prototype and refine our survey before receiving teacher feedback, and this effort was highly collaborative.

With multiple teammates, it was possible to divide this task into several areas that could be worked on independently and in parallel. We first decided on a basic structure  and strategized the division of labor. Carol worked on the text/content based on a logic diagram we crafted together. While Carol crafted this outline, I created a mockup version in Tripetto. Without access to finalized concept sketches, I took some poetic license.

Screenshot of 2nd iteration prototype survey

As Carol and I worked together to refine the text copy, Cat and Chris worked together to create images and descriptive text for our participants. Once all of this content was ready for Tripetto, we began doing test runs, trying to break the experiences. This revealed some quirks with Tripetto’s logic functions and some of the less apparent features.

There are a few honorable mentions; Tripetto has a lot of subtle features that we often take for granted in other online experiences. Things like placeholder text, required fields, multiple choice radio buttons, checkboxes, multi and single-line text boxes. During the refinement phase, these features became essential and it was exciting to discover them—only after they were deemed essential enough to be worth the effort.

The minimalist UI of Tripetto made these features less evident, but not too hard to locate or execute. From start to finish, this experience felt a little shaky and uncertain but viable.

I often found myself this week grinding away on the platform, slipping into a state of mind that Mihaly Csikszentmihalyi describes as “flow.” In other words, creating a survey on Tripetto wasn’t easy to use, but just challenging enough to keep me interested in working through obstacles. I think that what helped support this effort the most was building models within platforms where everyone on the team is already fluent. For us, this was primarily Miro, Google Docs, and Sheets.

Screenshot of two representations of the survey, carried across platforms (Miro and Sheets)

First impressions matter, and we didn’t want to put out anything that wasn’t necessarily a work in progress. Even with this in mind, we did have a few last minute tweaks as we adapted our survey to maximize pulling power with other social media environments.

Arnold Wasserman’s desk critique was incredibly valuable for our team, as his feedback helped us to consider the importance of our survey as a communication tool. He recommended that we make the implicit, explicit, to directly communicate to our participants what we expected and why. We were encouraged to explain what questions we were asking, and to share this openly. This kind of transparency can be tedious, especially in text-based systems. I took this to task and simplified statements throughout the entire experience.

This gave the survey a personality all its own; like a casual and curious friend, we asked about specifics but with little pressure. We kept things open.

Open data cannot be calculated, it must be evaluated for patterns. Next week will be a scramble to synthesize patterns and new insights as we work to finalize system concepts into well defined parameters. We hope that through this process we will also identify opportunities to produce relevant and compelling artifacts (our final output/deliverable).

It still feels like a risk to be so far into a process and to still not have a clear idea of what it is we are making. We instead draw our assurances from what we have already made: an index of relevant articles, interview notes, countless diagrams and visual representations of high-level abstract concepts and maps at almost every level of visual fidelity imaginable, hundreds of presentation slides, dozens of pages of reflective text, and months worth of slack messages, shared links, and drafted emails. We created interactive digital workshop spaces and protocols for our participants, and archives with 256-bit encryption.

When looking at the collective volume of effort from this team, it’s difficult to imagine that we wouldn’t make something meaningful in the end. Is that too optimistic? Ask me in a month.