On false augmented agency and what surveillance capitalism and user-centered design have to do with it
Main Article Content
Abstract
In the last years, there has been a surge in AI-powered products. Often marketed as "free", these services operate as hooks to lure unsuspecting users into voluntarily giving up data about every aspect of their life. Their data is the primary fuel of surveillance capitalism, a new economic system that exclusively benefits so-called Big Tech organisations at the expense of personal privacy and freedom of choice. This paper argues the ways these AI-powered products are being imagined and designed is further generalising a kind of, "enframing" that encourages a bureaucratic relationship with the world disguised as (a false sense of) augmented agency. This paper shows that technologically informed philosophical reflections can contribute to getting ourselves back into the feedback loop of technological mediation by helping us recognise our "becoming" with technologies as a design process.
Downloads
References
Baek, E., Cagiltay, K., Boling, E., & Frick, T. (2007). User-centered design and development. In J. M. Spector, M. D. Merrill, J. van Merriënboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 659–670). New York; London: Lawrence Erlbaum Associates, Taylor & Francis Group.
Bogost, I. (2012). Alien phenomenology, or what it’s like to be a thing. Minneapolis, Minnesota: University of Minnesota Press.
Bohman, J. (2016). Critical theory. In E. N. Zalta (Ed.), The stanford encyclopedia of philosophy (Fall 2016). https://plato.stanford.edu/archives/fall2016/entries/critical-theory/; Metaphysics Research Lab, Stanford University.
Brown, W. (2015). Undoing the demos: Neoliberalism’s stealth revolution. Brooklyn, New York: Zone Books (Distributed by the MIT Press).
Bunnin, N., & Yu, J. (2009). The Blackwell dictionary of Western philosophy. Oxford: Wiley- Blackwell.
Campbell-Kelly, M., Aspray, W., Ensmenger, N., & Yost, J. R. (2014). Computer. A history of the information machine (3rd ed.). Boulder, Colorado: Westview Press.
Cudworth, E., & Hobden, S. (2014). Liberation for straw dogs? Old materialism, new materialism, and the challenge of an emancipatory posthumanism. Globalizations, 12(1), 134–148. https://doi.org/10.1080/14747731.2014.971634
Danaher, J. (2018). Toward an ethics of AI assistants: An initial framework. Philosophy & Technology, 31(4), 629–653. https://doi.org/10.1007/s13347-018-0317-3
Fischer, M., & Taub, A. (2019, August 11). How YouTube radicalized Brazil. Retrieved 24 September 2019, from The New York Times website: https://www.nytimes.com/2019/08/11/world/americas/youtube-brazil.html
Floridi, L. (2002). On the intrinsic value of information objects and the infosphere. Ethics and Information Technology, 4, 287–304. https://doi.org/10.1023/a:1021342422699
Floridi, L. (2005). The ontological interpretation of informational privacy. Ethics and Information Technology, 7(4), 185–200. https://doi.org/10.1007/s10676-006-0001-7
Floridi, L. (2007). A look into the future impact of ICT on our lives. The Information Society, 23(1), 59–64. https://doi.org/10.1080/01972240601059094
Floridi, L. (2012). The road to the philosophy of information. In H. Demir (Ed.), Luciano Floridi’s philosophy of technology: Critical reflections (pp. 245–271). https://doi.org/10.1007/978-94-007-4292-5_13
Floridi, L. (2013). Technology’s in-betweeness. Philosophy & Technology, 26(2), 111–115. https://doi.org/10.1007/s13347-013-0106-y
Floridi, L. (2014). The fourth revolution: How the infosphere is reshaping human reality. Oxford, UK: Oxford University Press.
Floridi, L. (2019). Marketing as control of human interfaces and its political exploitation. Philosophy & Technology, 32(3), 379–388. https://doi.org/10.1007/s13347-019-00374-7
Flusser, V. (2011). Into the universe of technical images (K. N. Hayles, M. Poster, & S. Weber, Eds.; N. A. Roth, Trans.). Minneapolis: University of Minnesota Press. (Original work published 1985)
Flusser, V. (2014). Gestures (N. A. Roth, Trans.). Minneapolis: University of Minnesota Press. (Original work published 1991)
Foster, J. B., & McChesney, R. W. (2014). Surveillance capitalism: Monopoly-finance capital, the military-industrial complex, and the digital age. Monthly Review, 66(03). Retrieved from https://monthlyreview.org/2014/07/01/surveillance-capitalism/
Galanos, V. (2016). Floridi/Flusser: Parallel lives in hyper/posthistory. In Synthese Library (Studies in Epistemology, Logic, Methodology, and Philosophy of Science): Vol. 375. Computing and philosophy: Selected papers from IACAP 2014 (pp. 229–243). https://doi.org/10.1007/978-3-319-23291-1_15
Galanos, V. (2017). The double meaning of ‘replacement’ and the moral value of human and nonhuman inforgs: Crossroads of philosophy of information and actor-network theory. In P. Arvola, T. Hintsanen, S. Kari, S. Kolehma, S. Luolin, & J. Sillanpää (Eds.), Improving quality of life through information. Proceedings of the xxv Bobcatsss Symposium (pp. 79–84). Tampere, Finland.
Garber, M. (2014). These dolphins are using sea sponges as tools. The Atlantic. Retrieved from https://www.theatlantic.com/technology/archive/2014/04/these-genius-dolphins-are-using-sea-sponges-as-tools/361168/
Gadwalladr, C., & Graham-Harrison, E. (2018). Revealed: 50 million Facebook profiles harvested for Cambridge Analytica in major data breach. Retrieved from https://www.theguardian.com/news/2018/mar/17/cambridge-analytica-facebook-influence-us-election
Girardin, F. (2019, January 16). When automation bites back. Retrieved 10 September 2019, from Near Future Laboratory website: http://blog.nearfuturelaboratory.com/2019/01/16/when-automation-bites-back/
Graeber, D. (2012). Of flying cars and the declining rate of profit. The Baffler, (19). Retrieved from https://thebaffler.com/salvos/of-flying-cars-and-the-declining-rate-of-profit
Graeber, D. (2015). The utopia of rules: On technology, stupidity, and the secret joys of bureaucracy. Brooklyn; London: Melville House.
Heidegger, M. (1977a). The question concerning technology. In W. Lovitt (Trans.), The question concerning technology and other essays (pp. 3–35). New York; London: Garland Publishing. (Original work published 1954)
Heidegger, M. (1977b). The question concerning technology and other essays (pp. xxxix, 182; W. Lovitt, Trans.). New York; London: Garland Publishing. (Original work published 1954)
Ihde, D. (2009). Postphenomenology and technoscience: The peking university lectures (L. Langsdorf, Ed.). Albany, New York: SUNY Press.
Ihde, D., & Malafouris, L. (2018). Homo faber revisited: Postphenomenology and material engagement theory. Philosophy & Technology, 32(2), 195–214. https://doi.org/10.1007/s13347-018-0321-7
Jacobs, I. F., Bayern, A. von, & Osvath, M. (2016). A novel tool-use mode in animals: New caledonian crows insert tools to transport objects. Animal Cognition, 19(6), 1249–1252. https://doi.org/10.1007/s10071-016-1016-z
Johnson, D. G., & Verdicchio, M. (2017). Reframing AI discourse. Minds and Machines, 27(4), 575– 590. https://doi.org/10.1007/s11023-017-9417-6
Keynes, J. M. (2011). Economic possibilities for our grandchildren. In Essays in persuasion (pp. 358–373). New York; London: W. W. Norton & Company. (Original work published 1930)
Kittler, F. A. (1999). Gramophone, film, typewriter (T. Lenoir & H. U. Gumbrecht, Eds.; G. Winthrop- Young & M. Wutz, Trans.). California: Stanford University Press.
Kurtis, L. (2019, August 13). Apple locked me out of its walled garden. It was a nightmare. Retrieved 4 October 2019, from https://qz.com/1683460/what-happens-to-your-itunes-account-when-apple-says-youve-committed-fraud/
Martinho-Truswell, A. (2018, February 13). To automate is human. Retrieved 24 September 2019, from Aeon website: https://aeon.co/essays/the-offloading-ape-the-human-is-the-beast-that-automates
Mayer-Schönberger, V., & Cukier, K. (2013). Big data: A revolution that will transform how we live, work, and think. Boston; New York: Houghton Mifflin Harcourt.
McCoy, D. E., Schiestl, M., Neilands, P., Hassall, R., Gray, R. D., & Taylor, A. H. (2019). New caledonian crows behave optimistically after using tools. Current Biology, 29(16), 2737– 2742.e3. https://doi.org/10.1016/j.cub.2019.06.080
McNeil, J. (2018). Big brother’s blind spot. Retrieved from https://thebaffler.com/salvos/big-brothers-blind-spot-mcneil
Merwin, C., Wendland, A. J., & Hadjioannou, C. (2018). Introduction: Heidegger’s thinking through technology. In A. J. Wendland, C. Merwin, & C. Hadjioannou (Eds.), Heidegger on technology (pp. 1–12). Retrieved from https://books.google.pt/books?id=1QlrDwAAQBAJ
Mumford, L. (1967). The myth of the machine: Technics and human development. Harcourt, Brace & World.
Naughton, J. (2019, January 20). ’The goal is to automate us’: Welcome to the age of surveillance capitalism. Retrieved 1 February 2019, from https://web.archive.org/web/20190201222516/
Nelson, T. H. (1974). Computer Lib / Dream Machines: New freedoms through computer screens a minority report (1st ed.) [Self published]. Sausalito, California.
Norman, D. A., & Draper, S. W. (Eds.). (1986). User centered system design: New perspectives on human–computer interaction. New Jersey; London: Lawrence Erlbaum Associates.
O’Neil, C. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. New York: Crown.
Ortega y Gasset, J. (1964). Meditación de la técnica. In Obras completas: Vol. Vol. 5 (1933–1941) (6th ed., pp. 317–375). Madrid: Revista de Occidente. (Original work published 1939)
Pitt, J. C. (2011). Doing philosophy of technology. https://doi.org/10.1007/978-94-007-0820-4
Reagle, J. (2015). Following the joneses: FOMO and conspicuous sociality. First Monday, 20(10). https://doi.org/10.5210/fm.v20i10.6064
Ruckenstein, M., & Granroth, J. (2019). Algorithms, advertising and the intimacy of surveillance. Journal of Cultural Economy, 1–13. https://doi.org/10.1080/17530350.2019.1574866
Rosenberger, R. (2014). Multistability and the agency of mundane artifacts: From speed bumps to subway benches. Human Studies, 37(3), 369–392. https://doi.org/10.1007/s10746-014-9317-1
Thaler, R. H., & Sunstein, C. R. (2008). Nudge: Improving decisions about health, wealth, and happiness. Retrieved from https://books.google.pt/books?id=cYdYngEACAAJ
Thompson, S. A. (2019, April 30). These ads think they know you. Retrieved October 22, 2019, from The New York Times website: https://www.nytimes.com/interactive/2019/04/30/opinion/privacy-targeted-advertising.html
van Dijck, J. (2014). Datafication, dataism and dataveillance: Big data between scientific paradigm and ideology. Surveillance & Society, 12(2), 197–208. https://doi.org/10.24908/ss.v12i2.4776
Varghese, S. (2019, October 21). The junk science of emotion-recognition technology. Retrieved October 22, 2019, from The Outline website: https://theoutline.com/post/8118/junk-emotion-recognition-technology?zd=2&zi=xi4vmsgi
Wallach, D., & Scholz, S. C. (2012). User-centered design: Why and how to put users first in software development. In A. Maedche, A. Botzenhardt, & L. Neer (Eds.), Software for people: Fundamentals, trends and best practices (pp. 11–38). https://doi.org/10.1007/978-3-642-31371-4_2
Weiser, M. (1999). The computer for the 21st century [Reprint]. ACM SIGMOBILE Mobile Computing and Communications Review, 3(3), 3–11. https://doi.org/10.1145/329124.329126 (Original work published 1991)
Yeung, K. (2016). ‘Hypernudge’: Big data as a mode of regulation by design. Information, Communication & Society, 20(1), 118–136. https://doi.org/10.1080/1369118x.2016.1186713
Zuboff, S. (2015). Big other: Surveillance capitalism and the prospects of an information civilization. Journal of Information Technology, 30(1), 75–89. https://doi.org/10.1057/jit.2015.5
Zuboff, S. (2019). The age of surveillance capitalism: The fight for a human future at the new frontier of power. New York: Public Affairs.
Zwier, J., Blok, V., & Lemmens, P. (2016). Phenomenology and the empirical turn: A phenomenological analysis of postphenomenology. Philosophy & Technology, 29(4), 313–333. https://doi.org/10.1007/s13347-016-0221-7