Project Comments turned off

ARt Explore

By on April 22nd, 2013

Final Project post by:

Andy Pruett, Anshul Bhatnagar and Mukul Sati

Overview and Lessons Learnt:
Our project was aimed at using AR to enhance the experience of viewing and interacting with large art-work such as murals and graffiti. Using natural image tracking for images of these sizes presents substantial challenges that we sought to solve, so as to allow for the creation of seamless augmented experiences.
Through the development of this project we gained an understanding of the types of interactions that are feasible/good and those that are not within our problem space. We were also exposed to some of the underlying details of tracking technology and were able to sucessfully achieve the goal that we set out to accomplish, protyping a couple of experiences that demonstrate seamlesss tracking of large images.

Problem solution co-evolution:
There is a body of work in cognitive problem-solving that talks about the problem solution co-evolution that any problem solving episode entails. In an exploratory project as ours, this impacted us greatly, and we continually refined our goals as well as our approaches.
We started with the idea of recovering lost murals, murals that were removed due to political/social reasons, thus allowing users to experience the temporal evolution of artwork at the location. This idea was adapted once we realized that there might not be enough features to track and identify the background which we are to augment with the removed artwork. GPS would fail too because of its inaccuracy and inconsistency at the scale that we are talking about. The new problem was, thus, to provide for continuous tracking of large images and artwork.
The issue with tracking large images such as murals is that the definition of features changes depending on how close or far the camera is from the target image. For example, when you are close to the mural, the features are sharp and pointed but the same features become curved or blend into the image when the camera is moved away from the target. The view might not also include sufficient features for the image to be successfully tracked. Coupled to this, Vuforia also places a limit on the file size of images that can be uploaded to the device database for tracking.

Our Approach

To solve these problem, we used a 2 step process:

  1. Divide the a high-res version of image into multiple grids and stitch them together to form 1 Mult-Image target.
  2. Use a low-res image of the target and add it to the Multi-Image target.

 

Creation of a multi-image target from a large image by sub-dividing into uniformly sized grids

Creation of a multi-image target from a large image by sub-dividing into uniformly sized grids

The first step helped us to segregate the image into sub-images, which not only helps tracking the image from close distances but also allows us to create spatially localized experiences which will be discussed later. The second step enhanced these experiences and allowed the image to be tracked from larger distances. We also developed a stand alone application that allows a user to automatically generate these sub-images and combine them to form a multi-image trackable target to assist in the creation of such experiences.

Now that we had a trackable target which works from all distances, we used the information about the image, the field of view of the iPhone camera and distance of the phone from the image and determine which all grids are visible to the user. This combined with the information about how far the user is from the tracked image allows us to recognize the user’s spatial context and what is visible to him. We use this information to retrieve the relevant content from the database. Thus we are able to recognize the Level-of-Detail that needs to be conveyed.

Schematic of the working of Level of Detail Augmentation

Schematic of the working of Level of Detail Augmentation

User Interactions

The user moves around in 3D space and using the above explained mechanisms his tracking is maintained and information about his position is explained. Now we use this information to identify his focus and level of detail and allow him to post comments or join an existing conversation.

Comment authoring and retrieval for Spatially Localized Augmentation

Comment authoring and retrieval for Spatially Localized Augmentation

This makes the experience more social and interactive. People can try to talk about certain localized features of the image or the entire image as a whole just by moving around in the space around them. We also experimented with in-situ display of such comment/augmentations to conserve screen real-estate:

insitu

 

Code, Video and Paper:

Code: https://github.com/indivisibleatom/artExplore

Video:  http://youtu.be/s5taQOzyBNo

Paper: http://www.cc.gatech.edu/~msati3/veFinal.pdf

Administrative Comments turned off

Progress Report II – AR for Participation with Outdoor Art

By on April 1st, 2013

artExplore Project Progress Report II:

Summary

Our Argon2 application called artExplore is developing towards our goal of supporting interactions with art or images of very large size in an outdoor setting. In this document we describe the evolving vision for the app, and report our proof-of-concept milestones currently working and in progress. In addition, we lay out our goals for the remaining time in the semester.

Current Deployment

The system is deployed at http://artexplore.aws.af.cm (not a typo, actually ‘cm’) using a Node.js backend, MySQL database, and multi-image-tracking filesets generated with Qualcomm Vuforia. The code for the current deployment is present at https://github.com/indivisibleatom/artExplore.

Demonstration Materials

Sub-image Multi-image Tracking Method, by Anshul.

http://www.youtube.com/watch?v=qxnyeYdej_0

Server Backend and showing comments based on estimating area of interest using multi-image tracking, by Mukul.

http://youtu.be/fmTIUaQtDes

Showing and Hiding Comments With Click Interaction, by Andy.

http://www.youtube.com/watch?v=sYyQuxHymRk

Content Creation Desktop App, by Mukul.

Project Comments turned off

Project Progress report – Stage 2

By on April 1st, 2013

As part of our proposed application to develop a Marauders Map AR application, we have made following progress:

1)      When the application loads, an overview of Marauder’s Map is displayed along with a CSS button which says – ‘Touch and Hold to take oath’. As the user holds the button, the oath appears on the screen. If the user releases the button before the oath gets over, it should be re-started again. When the oath is completed, the map is opened and displayed to the user

2)      The next screen of the map shows the tutorials associated with the game. It gives a walkthrough of the tasks to be performed by the user as part of the game. The user has to attain 3 horcrux by carrying out 3 tasks in different sections/rooms of the map.

3)      The user can navigate through these walkthrough screens by clicking on navigation buttons on each screen.

4)      When the user goes through the tutorials and starts the game, a pair of footsteps and wand being displayed on the map when the camera is held a static position. The footsteps identify the player of the game.

5)      As user moves the camera across the map in order to go to a particular area, the position is tracked and footsteps and wand move across the map, relative to the camera.

6)      The user has to move (in terms of footsteps) to three different rooms in the map to get a horcrux from each room. However, the rooms open sequentially and the user has to carry out the tasks in that sequence. So, at all time, the rooms which user can access are highlighted with green border while others with Red.

Features under development

1)      Spell selection module – In the attempt to attain the horcrux, user would require casting several spells given the context of the task. A slider menu on the left of the screen will have icons for all the available spells. The user can select a spell to view specific details about the spell. The user can use the spell by loading the spell to the wand. The spell will then be invoked when the user clicks the wand next. The menu and interface for spells has been developed and needs to be integrated with the application.

2)      3D data models – We plan to incorporate some 3D models for different scenarios/tasks of the game. For one scenario, we developed a 3D model of a treasure chest which contains a horcrux. The user would have to cast ‘Alohomora’ spell to open the chest and access the horcrux. The chest and the animation to open the chest were developed using 3DSMax. However, we ran into some challenges while integrating the 3D objects with Argon and plan to solve this issue on priority.

URL of the application:

http://www.prism.gatech.edu/~hchhabra3/ve/map.html

URL of the video:

http://www.youtube.com/watch?v=gfvm9r5erNg&feature=youtu.be

Team members:

Hitesh Chhabra

Vipul Thakur

 

 

 

Project Comments turned off

Progress Report I – AR for Participation with Outdoor Art

By on March 11th, 2013

Brief overview:

We are running a bit behind schedule due to certain issues revealed in prototyping, requiring us to try out different alternatives as opposed to our initial plans. As we see it,  should not affect the overall deliverable time-schedule, but we might have to rethink the user interaction so as to provide a best-possible experience in line with technical considerations.

Tasks laid out: 

  1. Prototype/explore registration issues, gather media, begin IRB approval process for User Studies.
  2. Registration of images in lab setting (backend, client side code, image placement). Meet with stake-holders for interaction ideation and wants/needs analysis, level of detail viewing.

Tasks completed: 

  1. Prototype/explore registration issues, gather media.
  2. Registration of images in lab setting. Meet with stake-holders for interaction ideation and wants/needs analysis, level of detail viewing .

Prototypes: http://cc.gatech.edu/~msati3/argon/multiMarkers/basic.html (multiple image markers simultaneous tracking) and http://cc.gatech.edu/~msati3/argon/frameMarkerTransform/index.html (explorations into using the rotation angles of the phone to determine viewing frustum in case of absence of frame-markers). As things stand, we believe we will be able to get the desired interaction we want using a multi-image target, as demonstrated by the prototype @ http://cc.gatech.edu/~msati3/argon/multiImage/basic1.html. Initial mural media @ https://www.dropbox.com/sh/epslvenxgcm4xip/HSodB1c-tP. Initial sync up with people involved scheduled.

Tasks not completed/still in progress:

IRB approval – pushed back till prototyping complete.

Risks update:

1.   Prototyping has informed us that the primary risk is figuring a way to track a large image target, which might not be completely in view all the time. We have approached this problem via subdivion of the target, sensor fusion, and presently, using a single multi-image target. We need to nail this work-item before proceeding.

Video: