Fragmented Layers of Design Thinking: Limitations and Opportunities of Neural Language Model-assisted processes for Design Creativity

From Design Computation
(Redirected from DCIO.2022.MMLW2640)
Jump to: navigation, search
DCIO2022-Logo.png
DC I/O 2022 Paper and slides by Emmanouil Vermisso. https://doi.org/10.47330/DCIO.2022.MMLW2640 | Watch Left | Left | Left


DCIO2022 S3-1 E-Vermisso.png


Abstract

This paper offers insights about the otherwise limited NLM-driven methodologies, supporting an examination of design creativity following the ‘process’ approach. [Abraham 2018] Recent application of AI models which rely on natural language processing (semantic references) is increasingly popular because of their directness and ease-of-use. Neural Language Models (NLMs) like VQGAN+CLIP, DALL-E, MidJourney) offer promising results, [Rodrigues, et al. 2021] seemingly bypassing the need for expensive datasets and technical expertise. Naturally, such models are limited because they cannot capture the multimodal complexity of architectural thinking and human cognition in general [Penrose 1989]. Alternative approaches propose the combination of NLMs with other artificial neural networks (ANNs) i.e. StyleGAN; CycleGAN which are custom-trained on domain-specific data. [Bolojan, Vermisso and Yousif 2022] Architects seek to expand their agency within such AI-assisted processes by controling the input encoding, so they can subsequently convert the generated outcomes to 3D models fairly directly. Still, AI models of computer vision like NLMs and GANs offer 2-dimensional output, which requires extensive decoding into 3-dimensional format. While this may seem severely constraining, it presents a silver lining when it comes to furthering design creativity. Designers are asked to scrutinize their methods from a cognitive standpoint, because these methodologies not only encourage, but demand thorough interrogation of the design intentionality, the design decision making factors and qualification criteria. Text-to-image correlation, on which NLMs rely, and their 2-dimensional output, ensure that certain important considerations are not circumvented. Instead of obtaining a 3D model, multiple possible -fragmented- versions of it are separately implied. Often, ‘fake’ images generated by the ANNs promote contradictory inferences of space, which require further examination. The hidden opportunity within the limited format of AI models echo Neil Spiller’s comments about the advantage of drawing over animation techniques twenty years ago: “Enigma is a creative tool that allows designers to see bifurcated outcomes in their sketches and drawings; it plays on the inability of drawings to faithfully record the distinct placement and extent of architectural elements”. [Spiller 2001] Comparing animations to static drawings, Spiller praised the drawing’s ability to hold “…an imagined past and an imagined future”. ‘Reading’ these results involves the (human) disentanglement of high and low-level features and consciously allocating their corresponding qualities for curation. The process of evaluating ‘parts-to-whole’ visual relationships is noteworthy because it depends on shifting our attention away from certain features, and an unconscious binding of visual elements. [Dehaene 2014] The philosopher Alain wrote that “The art of paying attention, the great art,…supposes the art of not paying attention…the royal art”. [Dehaene 2021]. According to neuroscientists, the brain uses attention as an amplifier and selective filter, during one of the three major attention systems (Alerting; Orienting; Executive Attention). [Dehaene 2021] Orienting our attention addreses what we focus on and what we don’t. Suppressing the unwanted information, through interfering electrical waves, is useful for processing the object of attention. Considering the ANNs’ results at ‘Gestalt’ level, we can structure the AI-assisted process to ensure low-level features (composition) is retained while enhancing high-level (detail) features (Fig.1a)

Keywords

Design Creativity, Design Process, Neural Language Models, Artificial Neural Networks, Semantics, Visual Features, Attention.

Bibliography

  • Abraham, A., 2018. The Neuroscience of Creativity. Cambridge: Cambridge University Press.
  • Boden, M., 2013. Creativity as a Neuroscientific Mystery. In: Neuroscience of Creativity. Cambridge, MA: MIT Press, pp. 3-18.
  • Bolojan, D., Vermisso, E. & Yousif, S., 2022. Is Language All We Need? A Query Into Architectural Semantics Using a Multimodal Generative Workflow. Sydney, CAADRIA.
  • Browning, J. & LeCun, Y., 2022. AI And The Limits Of Language. [Online] Available at: https://www.noemamag.com/ai-and-the-limits-of-language/?fbclid=IwAR2-FGbuo-wroRtc1XeCSAv_lBMdw4fgq7AEKU-P57HpbXt3jJIQmB1qlBM [Accessed 4 September 2022].
  • Dehaene, S., 2014. Consciousness and the Brain: Deciphering how the brain codes our thoughts. New York: Penguin Books.
  • Dehaene, S., 2021. How We Learn: Why Brains Learn Better Than Any Machine...For Now. New York: Panguin Books.
  • Dennett, D. C., 2017. From Bacteria to Bach and Back: The Evolution of Minds. New York: Penguin Books.
  • Graziano, M. S., 2019. Rethinking Consciousness: A Scientific Theory of Subjective Experience. 1st edition ed. New York, NY: W. W. Norton & Company, Inc.
  • Green, A., 2018. Creativity in the Distance: The Neurocognition of Semantically Distant Relational Thinking and Reasoning. In: R. E. Y ung & O. V artanian, eds. The Cambridge Handbook of the Neuroscience of Creativity. Cambridge: Cambridge University Press, pp. 363-381.
  • Hadamard, J., 1945. The Mathematician’s Mind: The Psychology of Invention in the Mathematical Field. Princeton: Princeton University Press.
  • Hassabis, D., 2018. Creativity and AI (The Rothschild Foundation Lecture). [Online] Available at: https://www.youtube.com/watch?v=d-bvsJWmqlc
  • Kenett, Y. N., 2018. Going the Extra Creative Mile: The Role of Semantic Distance in Creativity - Theory, Research and Measurement. In: R. E. Yung & O. Vartanian, eds. The Cambridge Handbook of the Neuroscience of Creativity. Cambridge: Cambridge University Press, pp. 235-248.
  • Manovich, L., 2022. AI & Myths of Creativity. AD Machine Hallucinations: Architecture and Artificial Intelligence, 92(03), pp. 60-65.
  • Penrose, R., 1989. The Emperor’s New Mind: Concerning Computers, Minds and The Laws of Physics. Oxford: Oxford University Press.
  • Poonyagomol, N., 2022. Categorization, Archetypes & Exemplars: How Our Brain Processes Knowledge. [Online] Available at: https://vimi.co/categorization-archetypes-exemplars-how-our-brain-processes-knowledge/[Accessed 2 September 2022].
  • Radford, A. et al., 2021. Learning Transferable Visual Models From Natural Language Supervision. [Online] Available at: https://arxiv.org/abs/2103.00020 [Accessed 3 September 2022].
  • Rodrigues, R. C., Alzate-Martinez, F. A., Escobar, D. & Mistry, M., 2021. Rendering Conceptual Design Ideas with Artificial Intelligence: A Combinatory Framework of Text, Images and Sketches. s.l., s.n.
  • Spiller, N., 2001. Towards an animated architecture against architectural animation. Architectural Design 150 Architecture+Animation, 71(02), pp. 82-85.
  • Vermisso, E., 2022. Semantic AI models for guiding ideation in architectural design courses. Bolzano, International Conference on Computational Creativity.
  • Zabelina, D. L., 2018. Attention and Creativity. In: R. E. Yung & O. Vartanian, eds. The Cambridge Handbook of the Neuroscience of Creativity. Cambridge: Cambridge University Press, pp. 161-179.
  • Zak, D., 2021. ‘Nothing ever ends’: Sorting through Rumsfeld’s knowns and unknowns. [Online] Available at: https://www.washingtonpost.com/lifestyle/style/rumsfeld-dead-words-known-unknowns/2021/07/01/831175c2-d9df-11eb-bb9e-70fda8c37057_story.html

[Accessed 23 February 2022].