Tools used in AIX 2021
The following list provides an overview of some of the tools and methods used during the workshop, but does not claim to be exhaustive.
- Communication and collaboration
- Text generation
- Image processing
- Music generation
- Methods for interaction
Communication and collaboration
Due to the COVID-19 pandemic we are forced to run this workshop online. We would have loved to invite you to the Makerspace of the SLUB – maybe next time. This time we rely on the following tools, which we already used extensively during our collaboration within the STEAM INC. project.
Zoom
Zoom is a well-known videoconferencing software with the ability to use screen sharing, chatting, and breakout rooms. We appreciate the simplicity and reliability of Zoom.
Miro
Miro is a software to support collaboration with the help of virtual whiteboards. The participants can interact in real-time, use text and images and create a visual representation of ongoing discussions. In fact, we experienced certain benefits compared to real whiteboards. We will provide the necessary access to Miro.
https://miro.com/app/dashboard/
Text analysis and text generation
Markovify
Markovify is a simple, extensible Markov chain generator. Right now, its primary use is for building Markov models of large corpora of text and generating random sentences from that. However, in theory, it could be used for other applications.
https://github.com/jsvine/markovify
GPT-2
Generative Pre-trained Transformer 2 (GPT-2) is an open-source artificial intelligence created by OpenAI in February 2019. GPT-2 translates text, answers questions, summarizes passages, and generates text output. [...] The GPT architecture implements a deep neural network, specifically a transformer model, which uses attention in place of previous recurrence- and convolution-based architectures.[*]
https://openai.com/blog/tags/gpt-2/
GPT-3
Generative Pre-trained Transformer 3 (GPT-3) is an autoregressive language model that uses deep learning to produce human-like text. [...] GPT-3's full version has a capacity of 175 billion machine learning parameters. [...] The quality of the text generated by GPT-3 is so high that it is difficult to distinguish from that written by a human, which has both benefits and risks. [*]
AIX 2021 can provide trial accounts for GPT-3 to every participant with up to 300 K tokens each. These accounts can be used even when the workshop has finished.
https://openai.com/blog/gpt-3-apps/
Image processing
Reflect
Reflect face swap is the first ever automated realistic face swapping app based on machine learning. Reflect does not simply copy/paste a face from one person to another, but generates a seamless face onto the other person, so it fits precisely, even if swapping is made between completely different people.
https://reflect.tech/faceswap/hot
DALL·E
DALL·E is a 12-billion parameter version of GPT-3 trained to generate images from text descriptions, using a dataset of text–image pairs. It has a diverse set of capabilities, including creating anthropomorphized versions of animals and objects, combining unrelated concepts in plausible ways, rendering text, and applying transformations to existing images.
https://openai.com/blog/dall-e/
TensorFlow
TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML powered applications.
https://playground.tensorflow.org/
Deep Dream Generator
DeepDream is a computer vision program created by Google engineer Alexander Mordvintsev that uses a convolutional neural network to find and enhance patterns in images via algorithmic pareidolia, thus creating a dream-like hallucinogenic appearance in the deliberately over-processed images. [*]
https://deepdreamgenerator.com/
GAN Playground
GAN Playground lets you play around with Generative Adversarial Networks right in your browser. Currently, it contains three built-in datasets: MNIST, Fashion MNIST, and CIFAR-10. GAN Playground provides you the ability to set your models' hyperparameters and build up your discriminator and generator layer-by-layer. You can observe the network learn in real time as the generator produces more and more realistic images, or more likely, gets stuck in failure modes such as mode collapse.
https://reiinakano.com/gan-playground/
Music generation
MuseNet
MuseNet is a deep neural network that can generate 4-minute musical compositions with 10 different instruments, and can combine styles from country to Mozart to the Beatles. MuseNet was not explicitly programmed with an understanding of music, but instead discovered patterns of harmony, rhythm, and style by learning to predict the next token in hundreds of thousands of MIDI files. MuseNet uses the same general-purpose unsupervised technology as GPT-2.
https://openai.com/blog/musenet/
Methods for interaction
AIX 2021 can benefit from various methods developed by STEAM INC. and predecessor projects.
STEAM
STEAM thinking is a process that fosters collaboration between the Arts (A) and the Sciences (STEM - science, technology, engineering, and mathematics). Essential to the STEAM approach are the following attributes (not exhaustive): interdisciplinarity, equal footing, process-oriented, holistic, radical openness, experimentation, curiosity, critical thinking, reflection
Design Thinking
Design Thinking originates from working processes and organizing principles related to designing. Various stakeholders with different backgrounds participate in Design Thinking processes. It has been adapted and applied to other fields although there is not yet a fixed definition of Design Thinking today. A catalog of design thinking methods for the higher education sector has been developed during the Erasmus+ project DT.UNI.