Propostas de Teses de Mestrado em Computação Gráfica ... · centro de investigação em...

ISCTE-IUL, Av. Forças Armadas, 1649-026 Lisboa, Portugal | Phone: + 351 210464023 | email: [email protected] | http://istar.iscte-iul.pt/

http://istar.iscte-iul.pt/

centro de investigação em ciências e tecnologias da informação e arquitetura

Propostas de Teses de Mestrado em

Computação Gráfica, Visão por Computador,

Realidade Virtual, Realidade Aumentada,

Interação Pessoa-Máquina

2015-2016

METI, MEI, MIG, MIASIC ISCTE-IUL

Prof. Miguel Sales Dias

[email protected]

Prof. Associado Convidado DCTI/ISCTE-IUL

Director I&D Microsoft

Prof. Sara Eloy

[email protected]

Prof. Auxiliar DAU/ISCTE-IUL

Diretora do Departamento de Arquitetura e Urbanismo do ISCTE-IUL


Area: Virtual Reality

Topic: Immersive Virtual Reality with Oculus Rift• Scope

• https://www.oculus.com/rift/

• Resolution 960 x 1080 per eye

• Refresh Rate 60 Hz

• Persistence 2 ms, 3 ms, full

• 100° Field of View

• HDMI 1.4b

• USB 2.0

• Gyroscope, Accelerometer, Magnetometer

• 1000 Hz

• Positional Tracking with Near Infrared CMOS Sensor (60 Hz)

• Description of Work • State-of-the-art assessment

• Use Oculus Rift @ ISTAR-IUL

• Test existing VR apps available in the Oculus Rift store, using 3D desktop @

ISTAR-IUL VR Lab

• Integrate Oculus Rift in in-house VR system

• Leverage existing apps of the store

• Design the user experience

• Develop an app using Oculus Rift

• Perform usability evaluation

• Write International/National conference paper

• Write thesis

• Development Environment• CaveHollowspace (CAVEH) @ ISTAR-IUL with FMOD 3D sound, based in

OpenSceneGraph

• Unity 3D

• C++, C# Microsoft Visual Studio

Location• ISCTE-IUL/ISTAR_IUL, Lisbon



Topic: Understanding emotions in virtual spaces, via multi-

sensorial stimulation and biometric sensingScope• Virtual Environments are becoming increasingly popular and one niche of application that is attracting attention is

related to Virtual Environments (VEs), since they allow a museum visitor or tourist to feel immersed in sites that are

otherwise unreachable or difficult to visit, or allow a user to visit private or public spaces, before they are built. In

designing these environments, vision, hearing, touch, smell and taste are what VEs try to simulate via computing

means, so that users can report the sense of having been present somewhere else other than the place they really

were. Understanding the emotion felt by the user when navigating through the space, is also a relevant feedback to

the architect of the space. In this context, evaluating the user´s emotion and if the user felt a physical presence in a

VE is an important factor to characterize it.

Description of Work• State-of-the-art assessment

• Use biometric/ Physiological sensing available @ ISTAR-IUL (heart beat rate, electrodermal resistance, EEG -

Emotiv, eye tracking) and RGB-D sensing. Use existing VR spaces (3D building and/or urban model) with 3D

auralization and 3D sound rendering (ambisonics) available @ ISTAR-IUL

• Use several methods to measure basic emotions (anger, disgust, fear, happiness, sadness and surprise) through

Behavioral and Physiological sensing, comparison or questionnaires.

• Use CaveHollowspace middleware (CAVEH) @ ISTAR-IUL based in OpenSceneGraph

• Perform usability evaluation in the context of students work at Architecture department @ ISCTE-IUL

• Write International/National conference paper & Thesis

Development Environment:• CAVE Hollowspace middleware based in OpenSceneGraph

• Kinect SDK, X-10 SDK, Emotiv SDK, FMOD, Processing, Biometric Sensors SDK (Plux, Ergoneers)

• Microsoft Visual Studio, C#/C++

• Blender, Matlab

Location• ISCTE-IUL/ISTAR-IUL, Lisbon

Ref: MSC Thesis: 3-D Sound Enhanced Presence in Virtual

Environments (António Sérgio Azevedo)



Topic: Understanding presence in virtual spaces, via multi-

sensorial stimulation and biometric sensingScope• Virtual Environments are becoming increasingly popular and one niche of application that is attracting attention is

related to Virtual Environments (VEs), since they allow a museum visitor or tourist to feel immersed in sites that are

otherwise unreachable or difficult to visit, or allow a user to visit private or public spaces, before they are built. In

designing these environments, vision, hearing, touch, smell and taste are what VEs try to simulate via computing

means, so that users can report the sense of having been present somewhere else other than the place they really

were. Understanding the emotion felt by the user when navigating through the space, is also a relevant feedback to

the architect of the space. In this context, evaluating the user´s emotion and if the user felt a physical presence in a

VE is an important factor to characterize it.


• Use biometric/ Physiological sensing available @ ISTAR-IUL (heart beat rate, electrodermal resistance, EEG -

Emotiv, eye tracking) and RGB-D sensing. Use existing VR spaces (3D building and/or urban model) with 3D

auralization and 3D sound rendering (ambisonics) available @ ISTAR-IUL

• Use several methods to measure presence: through Behavioral and Physiological responses, comparison or

questionnaires.

• Use CaveHollowspace middleware (CAVEH) @ ISTAR-IUL based in OpenSceneGraph

• Perform usability evaluation in the context of students work at Architecture department @ ISCTE-IUL

• Write International/National conference paper & Thesis

Development Environment:• CAVE Hollowspace middleware based in OpenSceneGraph

• Kinect SDK, X-10 SDK, Emotiv SDK, FMOD, Processing, Biometric Sensors SDK (Plux, Ergoneers)


• Blender, Matlab






Topic: Sonification (a form of auditory display, is the

use of non-speech audio to convey information or

perceptualize data) of Interactive Tasks in Virtual

EnvironmentsScope• Human-object and human-human interactions are often facilitated by computational means.

Interaction designers aim at designing such interactions to make them more effective and

pleasant. Sonic Interaction Design emphasizes the role of sound as a mediator of meaningful

interactions.


• Develop sonification techniques in 3D sound auralization simulation, to enhance tasks in a

Virtual Environment, such as discovery and path finding, including sounds of Body Motion,

Footsteps, Impact Sounds, Ancillary Gestures, Soundscapes

• Develop Visualization and techniques

• Compare 3D Sound and 7:1 Surround Sound

• Use CaveHollowspace (CAVEH) with FMOD 3D sound, based in OpenSceneGraph Sound

in a new lab Virtual Reality lab with Ambisonics @ ISTAR-IUL

• Write paper & Thesis

Development Environment:• CaveHollowspace (CAVEH) @ ISTAR-IUL with FMOD 3D sound, based in

OpenSceneGraph

• FMOD, Ableton Live, Max MSP, Nuendo, Kinect One


• OpenSceneGraph, Blender

• Microphones, Audio Techniques (recording and reproducing sound), Ambisonics


http://pedrolopesresearch.wordpress.com/2011/11/09/its201

1-augmenting-touch-interaction-through-acoustic-sensing/



http://pedrolopesresearch.wordpress.com/2011/11/09/its2011-augmenting-touch-interaction-through-acoustic-sensing/


http://www.rollingrains.com/2007/03/


Topic: Brain-Computer Interface with Emotive• Scope

• BCI - Brain-Computer Interface

• http://www.emotiv.com/

• Emotive Provides access to dense array, high quality, raw EEG data

• 14 EEG channel locations

• 9 axis sensor (3x gyro, 3x accelerometer, 3x magnetometer)

• Description of Work • State-of-the-art assessment

• Use Emotive sensor @ ISTAR-IUL

• Conduct research using APIs and detection libraries:

• Facial Expressions

• Blink, Left wink, Right wink, Furrow (frown), Raise brow (surprise),

Smile, Clench teeth (grimace), Glance left, Glance right, Laugh, Smirk

(left side), Smirk (right side)

• Performance & Emotional Metrics:

• Instantaneous excitement, Long term excitement, Frustration,

Engagement, Meditation, Interest/ Affinity

• Mental Commands:

• Neutral, any of up to 4 pre-trained items from a list of 13 labels

• Leverage existing apps of the store


• Develop an app using Emotive SDK to detect Facial Expressions, Emotional

States or issue Mental Commands



• Write thesis

• Development Environment• C++, C# Microsoft Visual Studio

• Location• ISCTE-IUL/ISTAR_IUL, Lisbon


• Scope• VR usually defines a set of computer technologies that can be used to create

simulated environments, with several levels of realism. CAVE (Cave Automatic

Virtual Environment) is a multi-projection immersive virtual reality environment

where projectors are directed to several walls of a room-sized cube. BIM (Building

information modeling) is a process involving the generation and management of

digital representations of physical and functional characteristics of a facility, using

standards for the definition and exchange of product data.

• Description of Work• State-of-the-art assessment

• Specify a system allowing to exchange BIM data with the existing CaveH

environment. (Architecture department @ ISCTE-IUL provides the 3D building

model)

• Extend in-house VIARModes system @ ISTAR-IUL

• Recognize gestures via RGB-D sensing (Kinect One)

• Use European Portuguese speech API

• Develop a real-time prototype

• Perform usability evaluation in the context of students work at Architecture

department @ ISCTE-IUL.


• Write thesis

• Development Environment• Revit or other BIM system

• CAVE Hollowspace middleware based in OpenScenegraph

• Blender

• Microsoft Visual Studio

• C++, Python, Kinect SDK

• Location• ISCTE-IUL/ISTAR-IUL, Lisbon


Topic: Use of BIM in an immersive virtual reality environment


• Scope• VR usually defines a set of computer technologies that can be used to

create simulated environments, with several levels of realism. CAVE

(Cave Automatic Virtual Environment) is a multi-projection immersive

virtual reality environment where projectors are directed to several walls

of a room-sized cube


• Improve the existing system for authoring and visualization of content for

the CAVE environment allowing the visualization of several architectural

representations of space, including simulating projected shadows and

lighting effects and performing real time sections in the 3D model.

• Architecture department @ ISCTE-IUL provides the 3D building model)

• Use CaveHollowspace (CaveH) software provided by ISCTE-IUL


• Perform usability evaluation in the context of students work at

Architecture department @ ISCTE-IUL.


• Write thesis

• Development Environment• CAVE Hollowspace middleware based in OpenSceneGraph

• Blender

• Microsoft Visual Studio C#/C++, Kinect SDK



Topic: Visualization in an immersive virtual reality environment


• Scope• VR usually defines a set of computer technologies that can be used to create

simulated environments, with several levels of realism. CAVE (Cave Automatic

Virtual Environment) is a multi-projection immersive virtual reality environment

where projectors are directed to several walls of a room-sized cube


• Specify a system allowing the edition (moving, stretching, scaling, erasing, etc)

of a 3D model of building, within immersive VR and using natural gestures and

speech interaction (Architecture department @ ISCTE-IUL provides the 3D

building model)

• Recognize gestures via RGB-D sensing (Kinect One)

• Use European Portuguese speech API

• Use CaveHollowspace (CaveH) @ ISTAR-IUL





• Write thesis

• Development Environment• CAVE Hollowspace middleware based in OpenScenegraph

• Blender


• C++, Python, Kinect SDK



Topic: 3D model edition in immersive virtual reality environment


Area: Human-Robot Interaction

Topic: Communication between elderly and

Robot @ Home

Description of Work • State-of-the-art assessment

• Specify an interactive experience between a robot and an elderly

person to assist @ home

• Develop a NAO app using visual programming with Choreographe

• Use NAO Robot @ ISTAR-IUL

• Leverage existing NAO apps https://www.aldebaran.com/en

• Interaction:

• Loudspeakers

• Microphones

• Video camera

• Infra-Red

• LEDs

• Sensors:

• Force Sensitive Resistors

• Inertial unit

• Sonars

• Joint position sensors

• Contact and tactile sensors


• Develop a NAOapplication



• Write thesis

Development Environment

https://www.youtube.com/watch?v=nNbj2G3GmAo

SDK C++ e Pyhton

Choreographe

App store

Location: ISCTE-IUL, Lisbon

https://www.aldebaran.com/en

https://www.youtube.com/watch?v=nNbj2G3GmAo


Area: Human-Computer Interaction

Topic: Visual Speech Recognition for European Portuguese using

RGB-D sensing - Handling less visible articulators: Tongue & Teeth

• Scope• Visual Speech Recognition allows recognition of speech in highly

noisy environments. It is not affected by the acoustic conditions

and noise and contains a large amount of complementary

information to the acoustic signal

• Description of work• State-of-the art assessment

• Use in-house European Portuguese visual speech database

developed by prior Msc Thesis (Helder Abreu) “Visual Speech

Recognition for the European Portuguese”, captured with RGB-D

sensing (KinectOne)

• Detect less visible articulators: Tongue & Teeth using Computer

Vision techniques

• Develop newarticulatory features for Tongue & Teeth

• Improve end-to-end Automatic Visual Speech Recognition

prototype based with the new articulatory features

• Conduct an usability study of the developed prototype

• Write thesis and national/international conference paper

• Development environment• Microsoft Visual Studio, C#/C++

• Kinect One sensor

• Location: ISCTE-IUL/ISTAR_IUL, Lisbon


• Description• Assess State-of-the art

• Develop and extend tools, algorithms and techniques for PSL -

Portuguese Sign Language gestures (facial expression and both hand

gesturing), from Human Gesture, Visual Speech and Body Motion

Capture, using RGB-D sensing (KinectOne for Windows)

• Use and extend existing in-house C# platform that does Automatic

Visual Speech recognition (Lip-Sync) and Automatic PSL gesture

recognition

• Build a training and testing database of captured gestures

• Test and evaluate the solution with a panel of deft users and compare

with the State-of-the Art

• Write International/National conference paper & thesis

• Development Environment• Microsoft Visual Studio, C#/C++

• Microsoft Kinect SDK for Windows

• Kinect One Sensor



Topic: Portuguese Sign Language Recognition from

Human Gesture, Visual Speech and Body Motion Capture,

using RGB-D sensing


• Description

Assess State-of-the art

Collect and build a database of motion data

Develop tools to annotate and segment motion data

Apply to a virtual animated conversational character, speaking

sign language.

Create tools to produce descriptive scripts for virtual characters

Perform usability evaluation of the system with deft speakers

Write International/National conference paper & thesis

• Developement Environment

Microsoft Visual Studio, C#

Blender 3D, WebGL, HTML5/Javascript

Microsoft Kinect SDK for Windows

• Location

ISCTE-IUL/ISTAR-IUL, Lisbon


Topic: Create Portuguese Sign Language Conversational Animated Virtual Characters

based on Recordings of Human Performance


• Description


Adapt existing pictogram database

Design an intelligent approach to convert

from pictures to speech synthesis

Implement an application prototype that

automatically synthesizes pictogram-

based sentences

Perform usability evaluation of the

system with autist users

Write International/National conference

paper & thesis


Microsoft Visual Studio, C#, Universal Apps

• Location



Topic: Pictogram-based Communication: Creating a bridge between pictures and

speech in European Portuguese


• Description


Explore/Analyze the best audio

features for facial animation

Design and implement a

coarticulation model for natural facial

animation of 3D characters

Perform a performance evaluation of

the system

Write International/National

conference paper & thesis


Microsoft Visual Studio, C#, C++,

Autodesk Maya

• Location



Topic: Speech Animation of 3D Characters: Design coarticulation model for European

Portuguese


Area: Image Processing & Computer Vision

Topic: Large RGB-D Full Object Dataset• Scope

RGB-D object datasets are an essential requirement for the computer

vision community, for tasks such as object categorization, instance

recognition and pose estimation. The availability of 3D object models

registered and grouped by category is a novel interesting R&D hypothesis

for the research community.

• Description of Work State-of-art (SOA) assessment

Collect household 3D objects using available ISCTE-IUL data collection

system, with Kinect One Sensor for Windows: per object , collect depth images,

camera pose and compute global point cloud

Develop a global polygon mesh reconstruction algorithm using SOA techniques

Develop an Azure-based database to host the collected objects

Evaluate state-of-the-art of object classifiers and develop an object recognition

algorithm

Leverage current experience of the team with Eurographics Workshop on 3D

Object Retrieval (2015) - Shape REtrieval Contest (SHREC’15) Track: Retrieval

of Objects Captured with Kinect One Camera

Organize an international completion (2016) in the framework of Shape

REtrieval Contest (SHREC) - http://aimatshape.net/event/SHREC

Write International/national conference paper & thesis

• Development Environment C++, Microsoft Visual Studio

Point Cloud Library

Artoolkit

http://www.cs.washington.edu/rgbd-dataset/

• Location• ISCTE-IUL, ISTAR-IUL, Lisbon

http://aimatshape.net/event/SHREC





Topic: Live 3D Object Detection using RGB-D

sensing

• Scope

“Object detection determines if a potential object is present in a scene

and its location.” Suetens et al. 1992

3D Object Detection is a challenging open research area mainly due

to multiple objects in scene, occlusion, high computational-cost and

scalability to learned objects. This project focus on developing

innovative solutions for the these problems

• Description of Work

State-of-the-art assessment

Develop methods for object segmentation & detection

Develop a real-time prototype using Kinect One

Integrate the prototype in the existing 3D Reconstruction,

Classification and Tracking framework @ ISTAR-IUL

Write International/National conference paper

Write thesis

• Development Environment

C++

Microsoft Visual Studio

Point Cloud Library

OpenCV




Topic: Tracking people’s movement in real space

• Scope• Tracking people’s movement in space is a common technique to understand

patterns of use and to inform architecture and urban design with real user’s

preferences. Tracking can be done manually (as the image in the bottom) or

by software applications (e.g. using cellphones location tracking

technologies).


• Specify a system allowing to track walking humans featured in video data

captured in static real space positions.

• Generate graphic flows of people’s patterns of movement in that space

(Architecture department @ ISCTE-IUL provides the specifications for

traking and the locations to record video)

• Design tangible interfaces to allow such interaction





• Write thesis

• Development Environment• C#


• Point Cloud Library

• OpenCV

• Location• ISCTE-IUL/ISTAR-IUL, Lisboa



Topic: 3D Object Instance Recognition and

Classification using RGB-D sensing

• Scope• The ability to recognize and classify a unknown object based

on similar ones is an emerging trend in the research comunity,

to solve problems attained to video surveillance and assisted

living. Typically the goals are (1) recognize the ID of known

object instances, retrieving information attached to it (e.g.

position, size, utilities) and (2) infer a classification

(category/class) to unknown objects.


• Specify, develop and test techniques for instance recognition

and classification

• Integrate the developed prototype, using Kinect One, in the

framework of 3D Reconstruction, Classification and Tracking

framework @ ISTAR-IUL

• Conduct a benchmark evaluation between the system

developed and other state-of-the-art techniques


• Development Environment• C++


• Point Cloud Library

• OpenCV


Classes

Instances of “Cereal

Box” class

Brain Flakes

Chex

Kellogg's


Area: Mobility Apps

Topic: Augmented Reality App for Windows 10• Scope

• AR is the provision of intuitive information presentation, which enhances

the perceiver’s situational awareness and cognitive perception of the real

world. Enhancement is achieved by placing virtual objects or information

cues into the real world. AR (Azuma 97): (a) combines real and virtual

environments; (b) is interactive in real-time; (c) is registered in 3D

• Description of Work• State-of-the art assessment

• Specify, develop and test an AR prototype for mobility (Windows 10)

based on FIRST (Fast Invariant to Rotation and Scaling feature

Transform) tracking system available @ ISTAR-IUL

• Improve existing in-house developed AR system Arch

• Conduct a benchmark evaluation between the system developed and

other state-of-the-art techniques

• Conduct an usability study of the developed prototype in Windows 10 with

a mobility AR application for: (i) a museum which acts like a complement

to the exhibition showing additional information, or ii) a system for city

map travel (Architect department @ ISCTE-IUL provides the 3D building

models and other elements)

• Write thesis and national/international conference paper

• Development Environment• C# and Win10 SDK

• Universal App framework


• OpenCV

• Location

• ISCTE-IUL

• ISTAR-IUL, Lisbon


Area: Computer Aided Design

Topic: Shape grammar applied to an architecture design

problem• Scope

• Shape grammars are “algorithmic systems for creating and understanding

designs directly through computations with shapes, rather than indirectly

through computations with text or symbols.” (Knight, 2000). A shape grammar is

a set of rules that apply step-by-step to shapes to generate a language of

designs. The mechanism used to generate solutions based on a shape

grammar is to a certain extent, similar to an expert system because it uses the

knowledge of an expert to the problem field and, by using a particular

methodology, provides a solution.


• Specify a shape grammar interpreter that uses shapes and shape rules to

generate designs (Architecture department @ ISCTE-IUL provides the

specifications of a shape grammar example)






• Write thesis

• Development Environment• C#


• Location• ISCTE-IUL/ISTAR-IUL, Lisboa

Propostas de Teses de Mestrado em Computação Gráfica ... · centro de investigação em...

Documents

Transcript of Propostas de Teses de Mestrado em Computação Gráfica ... · centro de investigação em...