Automated Door Attendant: Progress Log

• May 2005
  • Conference paper submitted to ICMI 2005.
• April 2005
  • Conference paper submitted to UIST 2005.
• March 2005
  • Conference paper submitted to UbiComp 2005.
• February 2005
  • Videoconference module testing successful.
  • Initial masters thesis submitted.
• January 2005
  • Writing of masters thesis in progress.
  • Updated videoconference code within ADA. (Testing in progress...)
  • Updated ADA, PSI and vconf_adv file to include additional flags for videoconference.
• December 2004
  • Writing of masters thesis in progress.
  • Experiments completed.
  • Implemented new videoconference code within ADA. (Testing pending.)
  • Updated PSI and vconf_adv file to include additional parameters for videoconference.
  • Fixed some bugs involving misinterpreted speech commands.
• November 2004
  • Writing of masters thesis in progress.
  • Resumed experiments for additional test subjects.
  • Resolved sound card conflict in which Sphinx and XawTV both required microphone audio input.
  • ADA Video completed.
• October 2004
  • Writing of masters thesis in progress.
  • Experiments completed.
• September 2004
  • Experiments in progress.
  • Writing of masters thesis in progress.
  • Sphinx system files on ADA machine were mysteriously corrupted, causing several days of debugging and troubleshooting before discovering the source of the problem. Sphinx was reinstalled, which remedied the situation. (Once again, thanks to Frank R. for his invaluable help.)
  • State-dependent grammars resulted in significant decrease in performance due to high memory requirements. Thus, an improved global grammar was used instead, using more intelligent probabilistic parameters.
• August 2004
  • Implemented state-dependent grammars into ADA and Sphinx (almost complete). (Huge thanks to Frank R.)
  • Consulted with Stephen S about implementing Bronto videoconferencing into ADA.
  • Grammar rewritten for state-dependent behavior (link).
  • Improved motion-detection for updated camera placement.
  • Improved speech command validation for day-time selection.
  • Fixed font-size issue, so that now all text label contents are displayed in their entirety.
  • Audio input/output levels on ADA machine adjusted.
  • Resolved the "refresh" issue using a perpetual GtkTimer. Screen now updated instantly upon receiving voice commands.
• July 2004
  • All pixmap file locations defined in prefs.h, and presence verified upon startup.
  • Facade integration improved so that avatar window dynamically resized at each state.
  • The ADA Logfile now keeps track of all speech input received.
  • Included a text label on the bottom panel which actively displays the recognized phrase heard by Sphinx.
  • Optimized communication between ADA and Sphinx. Eliminated extraneous data between transmissions.
  • Speech integration functional. ADA now responds to voice commands and jumps to the appropriate state. (Eliminated "via" program and integrated all parsing and handshaking directly into ADA.)
  • Facade (3D avatar) integration functional. Instead of a static image, ADA now launches an OpenGL window containing a "live" rendering of the avatar. Remaining tasks:
    • Eliminate OpenGL window border, and reattempt to integrate using GtkGlExt widget.
    • Dynamically update facial expressions by making use of available functions.
    • Possibly animate the motion of the avatar.
• June 2004
  • Found and tested "Dillo", a basic GTK+ web browser (poor HTML support) that can be integrated into ADA seamlessly. It is unable to read frames, style sheets, and other non-core HTML tags.
  • Integration of speech (in progress), despite some difficulties.
  • Updated 3D OpenGL avatar (Facade) code to facilitate integration with ADA. Expected functions include head motion, blinking, and possibly lip-movement. Attempted to make use of GtkGlExt widget, but encountered several problems. Thus, Facade will be called externally.
  • Improved filename management and file verification on startup. The "prefs.h" file contains the names & location of all files used in the system, and the system issues warnings if any of the files are missing.
  • Improved PSI form validation. This ensures that invalid entries cannot be made, which would otherwise cause file problems.
  • The message notification emails now include a hyperlink to the specific message file (avi/png).
  • The logfile format has been updated to reflect the state information in each entry.
  • The PSI has been updated:
    • Added document-browser and videoconference settings sections to the PSI. The 'docs_adv.txt' and 'vconf_adv.txt' files may now be updated via the PSI.
    • Improved feedback when saving changes to a settings group.
    • Increased visibility of appointment message links in schedule cells.
• May 2004
  • Defined a finite state machine description of the ADA, in which each possible screen is assigned a number constant. The reason for this design is to facilitate the ultimate synchronization with the speech recognition module.
  • Added videoconference option from main menu (non-functional). From the PSI, the professor will be able to determine whether this menu option is enabled, as well as the party with whom the user will engage in videoconferencing (in progress).
  • Added video-message recording option to appointment scheduler. When making an appointment, the user may leave a video message instead of writing a note.
  • Schedule code entirely rewritten:
    • Week shown is now always from Monday to Friday. This is due to the fact that the previous schedule displayed the next 5 available days, which was found to be unintuitive and unlike a traditional calendar.
    • More intelligent suggestion of appointment times. Fixed a few bugs, so now the system now searches the entire week when suggesting convenient appointment times, taking into account off-hours and other constraints. Attempts to suggest two appointments on different days, if possible.
    • Appointment descriptions may span multiple rows. Description string gets parsed and line breaks are positioned at the nearest whitespace, when appropriate. Then, centers & spreads the text across the appointment block in a legible manner.
    • When schedule is accessed on weekends (or late Fridays), then following week is shown. In this case, a "late Friday" means that the ADA is accessed on a Friday at a time at which there are no more available appointment slots (taking into account off-hours and other constraints).
    • When schedule is accessed mornings or evenings (off-hours), now works as expected. This resolves a bug which appeared in the last version which did not properly handle this scenario.
  • Document browser's homepage URL moved to prefs.h file. It will eventually be possible to change this value from the PSI (in progress).
  • GUI uses larger fonts where possible. This can be noticed in prompts, suggested appointment times, schedule cells, etc.
  • Improved instruction wording & terminology. (With suggestions from professor, SRE team, etc...)
  • Cleaned up & optimized code (A LOT!). Eliminated a few hundred lines of code, optimized loops, renamed variables, etc.
  • Added "Do something else" button in video record mode.
  • Repositioned buttons and labels to allocate room for additional features.
  • Replaced "Post-It" writing pad with a notepad. The new design indicates to the user that a name and email address should be inscribed within the note.

Last update: May 1 2005 (Michael Perez)