Integrating API’s (Part 1)

    A major part of this project (which we expected to be the most difficult) is integrating all these programming libraries that work well individually into a single application. This includes Mobile 5.0 (the latest SDK that comes with the Pocket PC), OpenGL ES (a minimized "version of OpenGL" for embedded systems), Klimt (which sits on top of OpenGL ES and extends it to resemble normal OpenGL), PocketHAL (provides access to streaming frame buffers), DSVideoCE (wrapper library around DirectShow for camera access), and (most importantly) ARToolkitPlus (calculates camera position and orientation relative to physical markers in real time).

    I have successfully built simple applications that use all of these libraries, but individually, which was sometimes a nontrivial task. I have put together an application that uses ARToolkitPlus and Mobile 5.0, but this only looks at a static image and returns the modelview matrix that represents the camera position and orientation. I assume that once the video feed is accessed, the buffer can be used in a similar fashion. I was able to build an application that uses both DSVideoCE and PocketHAL which just displays a stream from the camera (of course, we haven't received our cameras yet so that makes you wonder a bit). I've encountered difficulty developing these mobile applications using Visual Studio 2005 because some of the builds were developed on different IDE's or for different platforms.

   Another issue we've dealt with was just knowing what system we are using. Developers sometimes use the terms Windows CE x.x, Pocket PC 2003/x.x, Mobile x.x interchangeably and this causes a great deal of confusion. Today I tried to build a project with embedded Visual C++ 4 but there was a conflict because it was looking at Windows CE 5.0 binaries which isn't necessarily the same as what the Mobile 5.0 SDK provides (although it is powered by CE 5.0). We've been building our projects for the ARM4V architecture, but I discovered today that the processor onboard (Intel PXA270) belongs to the XScale generation architecture which is the 5th generation of ARM. I don't know yet what this means about the compatibility of the tools we are using.

Stages of Design

Overview

Our project encompasses the requirements analysis and design stages of design although much of our time will be spent on the technical aspect of design. We hope to have a working system that at least serves as the foundation of a growing project that enhances the education process using handheld augmented reality. One really interesting aspect of our project is its extensibility. Once we have a text-based system working, we can evaluate it and attempt to resolve whatever issues there are using other technologies.

Requirements Analysis

A primary issue in requirements analysis is assessing what can be done with the time we have. As mentioned before, the project can go in many directions because of its extensibility. We will be working on a text-based model that will lend itself to integrating other technologies such as speech recognition, various media formats, networking and GPS.
We are indebted to Daniel Wagner and those at the Graz University of Technology in Austria (http://studierstube.icg.tu-graz.ac.at/handheld_ar/) for their work in Handheld Augmented Reality. Their work on similar AR systems has given us insight on what is required for our project, namely, a series of API's that work together to set an event-driven programming design pattern that recognizes visual markers using handheld-mounted cameras. Despite this, due to differences in hardware and software, we foresee changes in requirements for example because we are using different mobile devices that have different operating systems and API's associated with them. We will also be using cameras that may need to be interfaced differently because of this.

Design and Evaluation
The bulk of the work that will be completed in this project this summer will be in design. We will need to integrate the API's for the mobile device, the graphics hardware, an image processing system and other "in-between" SDK's and assess their capabilities and limitations. These programming interfaces work well individually, but we've found and expect that there will be conflicts with them, especially since many of them were developed for more specific applications. While this is being resolved, my partner will be collecting the data that the application will be utilizing about the artworks, artists, etc. We will then create a written prototype of the system and develop an effective interface to the system using the principles we have been learning in this program. We will finally design experiments that evaluate the system as an assistive technology to education and an enhancement to the museum experience using human subjects. We expect it to have both qualitative and quantitative factors. There probably won't be enough time however to conduct experiments that generate sufficient data to provide solid evidence of our claim.

Project Overview

Problem: The educative process intrudes the enjoyability of the museum-going experience in standard text-based systems.

Claim: We believe our system will provide a non-intrusive interface that educates and enhances a walk through an art gallery.

Prospective Timeline

Week 4

• Resolve hardware/software issues, ordering new equipment
• Data gathering for application
• Integrating mobile, ARToolkit, DSVideoCE APIs
• Researching speech recognition engine and its capabilities
• Discussing interface models

Week 5

• Deciding on an interface model
• Developing a written prototype for interface
• Reading in a marker in an image on a handheld
• Reading in a marker on a camera (if equipment arrives)
• Learning sound API
• Collecting audio data from artist

Week 6

• Implementing the interface
• Application recognizes at least two patterns and responds through interface

Week 7

• Test runs in environment
• Reworking of interface model
• Optimizations
• Quick evaluation with REU participants

Week 8

• Future directions and extensions to project are discussed
• Working on presentation for symposium 

Problems with Dell Axim X50 and Pretec SmartCam

We've been having hardware/software issues with the Dell Axim X50 PDA and the Pretec Smartcam mobile SD camera that we're connecting to it. After doing a fresh boot on the X50, updating the OS (Mobile 5.0), ActiveSync and reinstalling the drivers and software that came with the Smartcam, I would get an error in device.exe upon closing Smart Camera and never get a feed from the camera. My partner, Jovan would get a feed, but the resolution would be less than what the setting claimed it was. Also, there was some pixelation distortion of stillshots taken with the camera.

I took Dr. McCrickard's advice and tried Jovan's camera on my PDA. Upon inserting the camera in the SD slot, it begins to flash, but the feed is never seen in Smart Camera. My camera works just as well as his on Jovan's PDA. We do not yet know whether to attribute the difficulties we're having to hardware (with the PDA or the camera) or software (with the OS or the camera software).

Usability Case Study Tool

In response  to the following prompt in your online journal in about 750 words: "Have you ever used like a case study in your own education?  What did you think of the tool presented?  What would you do differently? What else should be incorporated into a case study?"

    I have never used a case study before, much less one presented in a format like the Usability Case Study Tool available on the site (http://ucs.ist.psu.edu/default.asp). The tool provides a linear overview of the development process of different designs that were constructed to aid the study of usability. It starts from requirements analysis, works its way to design and finally usability testing (through common scenarios using each design).

    I think it would be wise to indicate somehow the depth of each case study and how much documentation there is for it. One only discovers this by traversing a hierarchy that makes all of them appear small initially. I would place at least a number next to each case study title indicating how many artifacts each contains. Even better, I would add a heading section that indicates how many artifacts each case contains, the different types, and allows the user to browse the artifacts by type. I might also add a search section since a user might be interested in a specific data that may be in the case study.

REU in HCI at Virginia Tech Orientation

Friday was the last day of orientation week at the REU in HCI program here at Virginia Tech. At the beginning of the week, participants and coordinators were just getting to know eachother. All of the participants are cool. We all come from different parts of the country and have similar senses of humor. I think we'll have much to learn, both in conducting research and in living with such diverse people. I already learned about how research is conducted, graduate life and Virginia Tech. I can't wait to dig into my project which deals with augmented reality and handheld computing.

Last weekend we all went to see the Roanoke Star (the largest man-made star in the world, but who's competing?), went to the movies (to sleep through X-Men 3), spent some money at Wal-Mart, played some volleyball and card games. We are also getting to know some of the people we dorm with who are part of the REU Chemistry at Virginia Tech. We're all having a great time and learning a lot about how research is conducted.