User-centred design actions for lightweight evaluation of an interactive machine learning toolkit
Main Article Content
Abstract
Machine learning offers great potential to developers and end users in the creative industries. For example, it can support new sensor-based interactions, procedural content generation and end-user product customisation. However, designing machine learning toolkits for adoption by creative developers is still a nascent effort. This work focuses on the application of user-centred design with creative end-user developers for informing the design of an interactive machine learning toolkit. We introduce a framework for user-centred design actions that we developed within the context of an European Union innovation project, RAPID-MIX. We illustrate the application of the framework with two actions for lightweight formative evaluation of our toolkit—the JUCE Machine Learning Hackathon and the RAPID-MIX API workshop at eNTERFACE’17. We describe how we used these actions to uncover conceptual and technical limitations. We also discuss how these actions provided us with a better understanding of users, helped us to refine the scope of the design space, and informed improvements to the toolkit. We conclude with a reflection about the knowledge we obtained from applying user-centred design to creative technology, in the context of an innovation project in the creative industries.
Downloads
References
Abras, C., Maloney-Krichmar, D., & Preece, J. (2004). User-Centered Design. In W. Bainbridge (Ed.), Encyclopedia of Human-Computer Interaction (Vol. 37, pp. 445–56). Thousand Oaks: Sage Publications. https://doi.org/10.1.1.94.381
Amershi, S., Chickering, M., Drucker, S. M., Lee, B., Simard, P., & Suh, J. (2015). ModelTracker: Redesigning Performance Analysis Tools for Machine Learning. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI '15, 337–346. https://doi.org/10.1145/2702123.2702509
Bernardo, F., Tanaka, A., Fiebrink, R., Parkinson, A., Meala, S., & Bevilacqua, F. (2015). D2.1 User-Centred Design Methodology. Retrieved from https://rapidmix.goldsmithsdigital.com/wp-content/uploads/2016/02/D2.1-User-Centred-Design-Methodology1.pdf
Bernardo, F., Zbyszyński, M., Fiebrink, R., & Grierson, M. (2017). Interactive Machine Learning for End-User Innovation. In Proceedings of the Association for Advancement of ArtificiaI Intelligence Symposium Series: Designing the User Experience of Machine Learning Systems (pp. 369–375).
Bevilacqua, F., Bernardo, F., Mealla, S., & Fiebrink, R. (2015). D2.2 Design Guidelines for Prototyping. Retrieved from http://rapidmix.goldsmithsdigital.com/wp-content/uploads/2016/02/D2.2DesignGuidelineforPrototyping.pdf
Blackwell, A. F. (2017). End-User Developers – What Are They Like? In F. Paternò & V. Wulf (Eds.), New Perspectives in End-User Development (pp. 121–135). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-60291-2_6
Caramiaux, B., Montecchio, N., Tanaka, A., & Bevilacqua, F. (2014). Adaptive Gesture Recognition with Variation Estimation for Interactive Systems. ACM Transactions on Interactive Intelligent Systems, 4(4), 1–34. https://doi.org/10.1145/2643204
Cherry, E., & Latulipe, C. (2014). Quantifying the Creativity Support of Digital Tools through the Creativity Support Index. ACM Transactions on Computer-Human Interaction, 21(4), 1–25. https://doi.org/10.1145/2617588
Chowning, J. M. (1973). The Synthesis of Complex Audio Spectra by Means of Frequency Modulation. Journal of the Audio Engineering Society, 21, 1–10. https://doi.org/10.1038/srep01061
Clarke, S. (2011). How Usable Are Your APIs? In A. Oram & G. Wilson (Eds.), Making Software: What Really Works, and Why We Believe It (1 edition, pp. 545–565). O'Reilly Media.
Cooper, A., Reimann, R., & Cronin, D. (2007). About Face 3: The Essentials of Interaction Design. Indianapolis, Indiana: Wiley.
Edwards, W. K., Bellotti, V., Dey, A. K., & Newman, M. W. (2003). Stuck in the Middle : The Challenges of User-Centered Design and Evaluation for Infrastructure. Chi 2003, 297–304. https://doi.org/10.1145/642611.642664
Fails, J. A., & Olsen, D. R. (2003). Interactive Machine Learning. In Proceedings of the 8th international conference on Intelligent user interfaces IUI 03 (pp. 39–45). https://doi.org/10.1145/604045.604056
Fiebrink, R., Cook, P. R., & Trueman, D. (2011). Human Model Evaluation in Interactive Supervised Learning. In Proceedings of the 2011 annual conference on Human factors in computing systems - CHI '11 (p. 147). https://doi.org/10.1145/1978942.1978965
Françoise, J., Schnell, N., & Bevilacqua, F. (2013). A multimodal probabilistic model for gesture-based control of sound synthesis. Proceedings of the 21st ACM International Conference on Multimedia - MM '13, 705–708. https://doi.org/10.1145/2502081.2502184
Hall, M. A., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009). The WEKA Data Mining Software: An Update. SIGKDD Explorations, 11(1), 10–18. https://doi.org/10.1145/1656274.1656278
Hartmann, B., Abdulla, L., Mittal, M., & Klemmer, S. R. (2007). Authoring Sensor-based Interactions by Demonstration with Direct Manipulation and Pattern Recognition. In Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '07 (p. 145). https://doi.org/10.1145/1240624.1240646
Holtzblatt, K., Wendell, J. B., & Wood, S. (2004). Rapid Contextual Design: A How-To Guide to Key Techniques for User-Centered Design. San Francisco, CA: Morgan Kaufmann Publishers Inc.
Kapoor, A., Lee, B., Tan, D., & Horvitz, E. (2010). Interactive Optimization for Steering Machine Classification. Proceedings of the 28th international conference on Human factors in computing systems - CHI '10. New York, New York, USA, NY, USA: ACM Press. https://doi.org/10.1145/1753326.1753529
Katan, S., Grierson, M., & Fiebrink, R. (2015). Using Interactive Machine Learning to Support Interface Development Through Workshops with Disabled People. In CHI '15: Extended Abstracts on Human Factors in Computing Systems 2015. https://doi.org/10.1145/2702123.2702474
Lieberman, H., Paternó, F., Klann, M., Paternò, F., & Wulf, V. (2006). End-user development: An emerging paradigm. End User Development, 9, 1–8. https://doi.org/10.1007/1-4020-5386-X
McPherson, A. (2017). Bela: An embedded platform for low-latency feedback control of sound. The Journal of the Acoustical Society of America. https://doi.org/10.1121/1.4987761
Monk, A. (2007). Lightweight Techniques to Encourage Innovative User Interface Design. In L. E. Wood (Ed.), User Interface Design: Bridging the Gap from User Requirements to Design. CRC Press LLC.
Myers, B. A., & Stylos, J. (2016). Improving API usability. Communications of the ACM, 59(6), 62–69. https://doi.org/10.1145/2896587
Nielsen, J. (1994). Usability Engineering. San Francisco, CA: Morgan Kaufmann Publishers Inc.
Norman, D. A. (2013). The Design of Everyday Things. MIT Press.
Norman, D., & Draper, S. W. (1986). User Centered System Design: New Perspectives on Human-Computer Interaction. Hillsdale, NJ, USA: L. Erlbaum Associates Inc.
Parkinson, A., Zbyszyński, M., & Bernardo, F. (2017). Demonstrating Interactive Machine Learning Tools for Rapid Prototyping of Gestural Instruments in the Browser. In Web Audio Conference 2017 (pp. 1–2). Retrieved from https://qmro.qmul.ac.uk/xmlui/handle/123456789/26150
Patel, K., Bancroft, N., Drucker, S. M., Fogarty, J., Ko, A. J., & Landay, J. (2010). Gestalt: Integrated Support for Implementation and Analysis in Machine Learning. In Proceedings of the 23nd annual ACM symposium on User interface software and technology (pp. 37–46). https://doi.org/10.1145/1866029.1866038
Ritter, F. E., Baxter, G. D., & Churchill, E. F. (2014). Foundations for Designing User-Centered Systems: What System Designers Need to Know about People. London: Springer London Heidelberg New York Dordrecht. https://doi.org/10.1007/978-1-4471-5134-0
Steiglitz, K. (1996). A Digital Signal Processing Primer: With Applications to Digital Audio and Computer Music. Addison-Wesley.
von Hippel, E. (1986). Lead Users: An Important Source of Novel Product Concepts. Management Science, 32(7), 791–805. https://doi.org/10.1017/CBO9781107415324.004
Zbyszyński, M., Grierson, M., & Yee-king, M. (2017). Rapid Prototyping of New Instruments with CodeCircle. In NIME 2017 Proceedings of the International Conference on New Interfaces for Musical Expression (pp. 227–230).
Zbyszyński, M., Grierson, M., Yee-king, M., & Fedden, L. (2017). Write once run anywhere revisited: machine learning and audio tools in the browser with C++ and emscripten. In Web Audio Conference 2017 (pp. 1–5). Centre for Digital Music, Queen Mary University of London.