Marek A. Perkowski, theatre director.

PORTLAND CYBER THEATRE. READING LIST FOR RESEARCHERS.

SECTION 5.1. BACKGROUND: GENERAL INFORMATION ABOUT HUMANOID ACTORS AND ROBOT THEATRES



  1. Robot Theatre: Theoreticians. Research Issues. Introduction. Development and learning. Developmental Psychology. Networks and Learning. Neuroscience. Open problems in robotic theatre. Theater as a metaphore for robotic research. Barbara Hayes-Roth and Virtual Theater at Stanford University. Oz Project at Carnegie Mellon University. Responsive Animated Characters.

  2. Robot Theatre: Robot Pageants. Example of robot festival. Summary of Session. Japanese robot contests. Robot exhibitions. International Robot Forums. Famous entertainment robot designers. Robot theatre sculptures-robots.

  3. Robot Theatre: Commercial Robot Puppets. Robotic puppets on vehicles. Heads. Remote controlled animals. Robot Theatre accessories. Complete literature references on Robot Theatre, from 2001.

  4. Robot Theatre: Oregon Cyber Theatre. Ideas for innovative robot theatre. Humans, Robots and Turing Test Theatre. Overview of theatrical robots. Evolvable Hardware for Intelligent Robots. Robot theatre at PSU in 2001. Presentation of robot puppets. Voting and agent behaviors. Social behaviors. Spider society. Game theory. Moral and Immoral Robots. Asimov's Laws and Ten Commandments for Robot world. The role of Internet and controlling humans. What people expect from robot theatre. Emotional robots. Complext dynamics. Image processing and robot vision. Robot development stages. Software architectures. Modes of operation. Research Plans.

  5. Robot Theatre: Humanoid Robots. Part 1. Why people build humanoid robots. What are humanoid robots good for? SRI report. Humanoids 2000 conference at MIT. Humanoid robots from Honda. Kismet, COG and japanese humanoids. Our first primitive humanoids. Evolutionary approaches to develop robot puppets.

  6. Robot Theatre: Humanoid Robots. Part 2. Honda robot. Bio-mimetic principles. Walking. Robots from KAIST. Sarcos robots. Hadaly-2. Japanese humanoids. Robot Band. Karate robot. Robonaut of NASA. Head and Arms. Architecture.

  7. Robot Theatre: Humanoid Robots. Part 3. Examples of various walking and humanoid robots from year 2002.

  8. COG architecture from MIT.
  9. KISSMET architecture from MIT.
  10. Robot Theatre. link to more details on humanoid robot projects.

  11. Robot learning emotional behaviors through immitation using Pattern Recognition and Image Processing. links to many robot emotions projects world-wide.
  12. Clayton Bayley robots.




SECTION 5.2. BACKGROUND INFORMATION ON BUILDING THE "FAITHFUL ROBOT" ROBOT THEATRE



This is a play for a robot and humans. The robot is a humanoid latex-face-type woman robot on wheeled platform. So far, the work on hands for the robot is done separately.
You can this this robot here.......

  1. "Faithful Robot" by Stanislaw Lem. Part 1.
  2. The robot who will be




SECTION 5.3. BACKGROUND INFORMATION ON FACE RECOGNITION AND BIOMETRIC TECHNOLOGIES.



One of the characteristics of our robot is the use of face recognition, facial gesture and body gesture recognition and voice recognition in interaction with humans. The robot collects and stores information about all people with whome it interacts.

RECOMMENDED MATERIAL ONLY FOR PEOPLE WHO WORK ON FACE DETECTION AND RECOGNITION.

  1. Biometrics Short Course. In PDF format. Face recognition overview. Advantages and obstacles. Applications. Related domains. Face detection. Eigenfaces - the basis for representation. Recognition with eigenfaces. Extensions to eigenfaces. Gabor Jets. Handling lighting and pose. Extending face recognition. Face forgery. Face recognition for watchlists. Face Databases. Speaker verification. Applications. Advantages. Obstacles. Domains. Features. Methods. Fixed text by Dynamic Time Warping. Text independent GMM. Cohorts/Universal background model. NIST evaluations. Lip motion verification. Signature verification. Signature acquisition. Signature forgery. Issues in Biometry. Multiple biometrics. Continuity of identity.

  2. Introduction to Face Detection and Recognition. In PPT format. What is face detection. Importance of face detection. Current state of research. Different approaches. Example. Face Recognition. What is it? Its applications. Different approaches. Example.

  3. Face recognition fundamentals. In PPT Format. Human faces. Face Image databases. References. Face Detection Methods. Clustering. Distance Measures. Deformable Face Template. Local and Global Transformations. Deformable model of facial features. Upper face action units. Lower face action units. Templates for various states. Features for action unit recognition. Classification from Feature Vector. Recognition rate. Appearance Model: landmarks on a face. Eigen-Vectors for Geometry and Photometry. Face Localization and Recognition. Hidden Markov Models. 4D space. Multi-scale detection. Edge features. Decision trees. Bounds analysis. Examples. Face Prior Learning.

  4. Discrete Cosine Transform for Face Recognition. Existing methods for feature extraction and advantages using DCT. Key characteristics of recognition systems. Information packaging using DCT. System description of DCT Recognition System. Brief information about ORL Database. Experimental Simulations.

  5. Bayesian Face Recognition Using Gabor Features. Basic Idea. Gabor features. Bayesian Analysis. Experiments on XM2VTS Database. Experiments on AR Database.

  6. Casino biometric solution by Cube. In PDF format. Description of practical commercial face recognition system for gambling industry.

  7. Dynamic face recognition. In PPT format. Purpose of this approach. History of face recognition. Problems. Solution: Apollo, components, face recognition technology used. Motion detection. Future work.

  8. Face Recognition and psychology. In PPT format. Representing individual faces. Evidence for holistic processing. Orientation is important. The Thatcher illusion. Pigmentation and shading. Cognitive neuropsychological evidence suggests for independent modules. Different models for face recognition. Information processing models. Interactive Activation and Competition (IAC) model of concept learning. Jets and Sharks model. IAC model by Burton.

  9. Effects of attractiveness and priming on face recognition.

  10. Using Hidden Markov Models for Face Recognition. In PPT format. Motivation behind this approach. Markov Chains - how to estimate probabilities. What is Hidden Markov Model (HMM). Embedded HMM. Observation Vectors. Training of face models. Face Recognition.

  11. Image Databases for Face Recognition Systems. In PPT format. Facial scans. Feature vector. Color Histogram. Graph (shape of face). Hashing. Indexing. Image Data Flow.

  12. HID Project. CMUs' HID (Human Identification) Project. State of the art in Face Recognition. Face recognition with non-cooperative subjects. CMU "Pie" Capture setup. Pose Invariant Face Recognition.

  13. Introduction to Pattern Recognition with applications to Face Recognition Problem. Face Analysis. Introduction to Pattern Recognition. Examples of Patterns. Two Schools of thinking. Features and Distributions. Main Issues in Pattern Recognition. What is a Pattern.

  14. Principal Component Analysis (PCA) Approach to Face Recognition. Rotate Coordinate System. Principal Component Analysis. Correlation Matrix. Linear Subspace. Receptive Fields. Non-rigid constrained spaces. Manifold Learning. Feature/Shape Models. Linear Discriminant Analysis. Eigenfaces vs Fisherfaces. Basis Shape Algorithms.

  15. Real Time (RT) face recognition. In PPT format. Why real-time face recognition. Why difficult. How done? Eigenfaces. Other face algorithms. What is wrong? Future of research.

  16. Unified Subspace Analysis for Face Recognition. In PDF Format. Face Difference Model. Unified Framework. PCA. Bayesian Face Recognition. Intrapersonal Subspace. Linear Discriminant Analysis. LDA subspace. Compare different subspaces. Unified Subspace Analysis. Experiments.

  17. Automatic Face Recognition Using Color Based Segmentation and Intelligent Energy Detection. Basic System Summary. Use of color analysis. Removal of regions. Morphological Processing. Template design. Matched Filter Operation. Face Detection steps and Progressive Masking. Results, conclusions, references.

  18. Subspace representation for Face Recognition. Four different subspace representations: PCA, PPCA, LDA and ICA. Kernel vs Non-Kernel. Two Data Bases with 3 different variations: Pose, Facial Expression, Illumination.

  19. Face Perception. Can you recognize upside-down faces? Illusions. Hollow face effect. The ambiguity of Shape-from-shading. Do our visual systems process faces in a special way. Perception as hypothesis. Measuring the strength of the hollow face Illusion. Perception of chimeric faces. The Fusiform Face Area. Mirror-reversed faces.

  20. Face Recognition Sensitivity. Face Recognition is very orientation-sensitive. Rocks' hypothesis. Detection of featural changes. Detection of configural changes. Integrative Model. Role of Parts and Configural Information. Parts in face recognition. Familiar vs unfamiliar face recognition. Independent or convergent processing? Computational modeling. Matching by correspondences. Component and configural processing. Matching under large view rotations.3D motion tracking. Texture manipulation and rendering. Task and procedure. Smiling faces. Talking faces.

  21. Recognition. Image as a feature vector. Recognition architectures. Recognition Scheme. Face Recognition. Eigenfaces and PCA. Illumination variability.

  22. Object Recognition using Pictorial Structures. Pictorial Structures. History. Part-Based Approaches. Formal Model Definition. Probabilistic models. Bayesian Approach. Generic Face Model. Learned 9 part face model. Generic Person Model. The Minimization Problem. Minimizing over tree structures. Classical Distance Transforms. Generalized Distance Transforms. Algorithm for MAP Estimate. Recognizing Faces. Recognizing People. Variety of Poses.

  23. Man Machine Interface. Processing Scheme. Human Visual System. Video Representation. Progressive and Interlaced Scan. Chrominance subsampling. Color Spaces. Object Detection - Face Detection Overview.

  24. Context Aware Computing.

  25. Vision based interface for pervasive computing. Finding faces. Tracking Pose. Hand Gestures. Full Body Interaction.

  26. Commerce Security. Access Control and Site Security techniques. Overview.

  27. Visual Authentication for Small Wireless Devices Build in Java.

  28. Human Semantic Memory.

  29. Perception.

  30. Disorders of Pattern Recognition.

  31. Introduction to Biometrics.

  32. Video Surveillance, Biometrics and Privacy After 9-11.

  33. Security Data Sheet.

  34. Introduction to Biometrics.

  35. Introduction to Biometrics and Authentication.

  36. Biometrics must be banned.

  37. Human-robot interaction. Very good overview.

  38. Person Recognition.

  39. VC07.pdf

  40. computer-vision.pdf

  41. facial-expression-and-identity-medium.ppt

  42. identification-speech.pdf



SECTION 5.4. NATURAL LANGUAGE PROCESSING AND SPEECH COMMUNICATION SYSTEMS.


Background Information and Probability
  1. A Survey of Probability Concepts
  2. Probability. Bayes Theorem.
  3. Fundamentals of Probability
  4. Fundamentals of Natural language Processing from point of view of probability
  5. Probability in Natural Language Processing

After reading this material you have to REALLY understand concepts of probability and conditional probability, Bayes Theorem and how to use decision trees in relation to conditional probabilities. I will give you problems to solve.

    Background information on Transforms.
  1. Signals
  2. Frequency and Bandwidth
  3. Fourier Transforms and Spectra
  4. Discrete Fourier Transform

After reading this material you have to REALLY understand concepts of Fourier Transform and Fast Fourier Transform and spectrum. I will give you problems to solve.

Background on Speech Analysis and Synthesis
  1. Introduction to Speech Processing
  2. Time and frequency domain
  3. Speech Acoustic Analysis
  4. Speech Synthesis

After reading this material you have to understand the fundamental concepts of speech processing.

SECTION 5.5. ROBOT LEARNING BY CONSTRUCTIVE INDUCTION


Topics: Machine Learning based on logic synthesis methods for a humanoid robot to immitate human behaviors.
Team Leader: Stefan Gebauer.
Literature:
  1. Papers about Ashenhurst/Curtis and Bi-Decomposition that I showed you in my Publications section.
  2. Report of Uland Wong about software for Big Ugly Greeter robot.
  3. Reports of Perkowski about robot theatre talking heads from Japan and Korea.
  4. Marek Perkowski, Rahul Malvi, Stan Grygiel, Mike Burns, Alan Mishchenko, ``Graph Coloring Algorithms for Fast Evaluation of Curtis Decompositions,'' Paper from DAC 1990 in PDF format.
  5. Slides from my WWW Page about Machine Learning.
  6. This is basically a software integration/system testing project.

Software to be used:
  1. Software written by Uland Wong and Alan Mishchenko in C++ and Visual Basic. For BUG robot.
  2. Software written by Atsumu Iseno for Professor Perky robot.
  3. Software from of Intel and CMU for robot vision and image processing. Only for vision part.
  4. Fonix tool for speech recognition and speech synthesis.
  5. Various classes developed by Martin Lukac.
  6. Alice like software for dialog. It may be in LISP.

  7. Below you will find more information and project ideas.

SECTION 5.6. GOALS OF THE MACHINE LEARNING CONSTRUCTIVE INDUCTION PROJECT.

  1. Understand the concepts of learning by decomposition.
  2. Understand, after Uland Wong, "from outside", how to use MVSIS in robot learning.
  3. Resurrect mechanically-electrically-software the robot head project of Uland Wong.
  4. Resurrect mechanically-electrically-software the robot head project - Professor Perky - of Atsumu Iseno.
  5. Link these projects together.
  6. Experiment with various logic synthesis methods for strongly unspecified functions and relations for the task of teaching some behaviors to the robot.
  7. The main research questions to be answered and documented in the report are the following:
    1. What are the reasons of learning errors when one uses the method of bi-decomposition of Boolean Relations?
    2. How the method of bi-decomposition of Boolean Relations can be improved for better learning results?
    3. Is it better than standard multi-valued decision tree for learning?
    4. What other algorithms included in MVSIS can be used for learning, do they give any advantages?
    5. These issues can be discussed for each of the following types of learning:
      1. Learning simple movements.
      2. Learning facial expressions.
      3. Learning reasonable dialogs.
      4. Learning from sensor information.
      5. Learning from vision.
      6. Learning from speech.
      7. Combined Learning, any of the above.
  8. Write a report for the class.


  • The task that are related to Theses of Stefan and Myron and not related to this class are:
    1. For Myron and Robert: Image processing/robot vision for human face recognition (who is it?), human mood recognition (smiling, angry, sad) and gestures recognition (waving hand, showing two fingers, showing five fingers, showing a fist, etc).
    2. For Stefan: Word Spotting using FONIX.
    3. For Stefan: Text to Speech using FONIX.
    4. For Myron, Stefan and Aminul Islam: Integrate Eliza/Alice like dialog programs to FONIX. Use Fonix tool's script language. Read the report of Hung Nguen about Fonix.
    5. For Stefan and Myron: Language for emotional robot behaviors.
    6. For Martin Lukac, Stefan, Aminul, Robert and Myron: Integrate vision, speech, dialog and robot movements.
    7. Basic Combinational Problems. Coloring, covering, cliques, decomposition.
    8. Basic Combinational Problems. Coloring, covering, cliques, decomposition. II
    9. Basic Combinational Problems. Coloring, covering, cliques, decomposition. III


    SECTION 5.7. EXAMPLES OF USING FONIX TOOLS FOR SPEECH RECOGNITION AND SYNTHESIS.

    The files that are executable or in FCB format you can download but you cannot view them.
    1. Report in Word format.
    2. Version 1.0. of report in Word format.
    3. Version 1.1. of report in Word format.
    4. PowerPoint Slides about statistics, necessary for the project on speech recognition evaluation.

    5. Example of Fonix script in FCB format. Dialer Demo.
    6. Example of Fonix script in FCB format. Cat and Dog.
    7. Example of Fonix script in FCB format. Direction.
    8. Another version of Fonix script in FCB format. Direction1.0.
    9. Another version of Fonix script in FCB format. Direction1.1.
    10. Another version of Fonix script in FCB format. Direction1.2.

    11. PSU executable.
    12. Direction executable.
    13. Prog. Exe

    14. Manual in Word format.
    15. Error Data in Word format.