powered by FreeFind

 

 

History of XmdvTool

The early years: 1993-1999

XmdvTool originated in 1993, when Matt Ward decided to merge several small programs he had written to visualize multivariate data in order to evaluate the effectiveness of each. The original techniques included were parallel coordinates, scatterplots, and star glyphs. He later added dimensional stacking, a technique he and two of his students (Jeff LeBlanc and Rajeev Tipnis) had created and implemented in a tool called N-Land. The techniques were linked via a highlighting mechanism controlled by N sliders, one for each dimension. Version 1.0 was developed using Athena Widgets and the Widget Creation Library. It had about 1000 lines of C code and about 600 lines of a resource file to define the interface. It was released to the public domain during the summer of 1994, and described in a paper at the Visualization '94 conference.

One of the reviewers of this paper commented that the program really needed direct manipulation to support the selection/highlighting process. Thus Ward and Allen Martin, a WPI graduate student, added functionality to support interactive specification and modification of the selection region, called the "brush", on the data displays themselves, rather than via separate widgets. This work formalized the notion of an N-dimensional brush, as opposed to the screen-space techniques that had been used before. By working in data-space rather than screen-space, users could specify a hyperbox of interest, which could then be highlighted, deleted, or masked. Also, the concepts of a ramped brush, which allowed points to belong to a brush in varying degrees, and a composite brush, built from logical combinations of existing brushes, weredeveloped. This work was reported at Visualization '95, and version 2.0 of the system was released that year. The code had grown to approximately 10,000 lines of C code and 1000 lines of resource file.

Because the source code was distributed free-of-charge, several researchers made their own variations on the code. Torsten Timm, a student at the University of Erlangen, released the unofficial version 3.0 in 1996. His extensions allowed data-driven glyph positioning, labeling of data points, and other enhancements.

Version 3.1, developed at WPI, included many features beyond those in Version 2.0, including interactive enabling and disabling of dimensions, arbitrary zooming, extended Tukey box plots for visualizing statistics of selected data, a reconfigured user interface, and an improved help facility. It was released in 1997.

In 1998, Matt Ward and Elke Rundensteiner received a 3-year NSF grant to investigate ways of visually exploring large multivariate datasets. The award, NSF:IIS 9732897, is being used to fund 2 graduate students: one in visualization and the other in databases. The key concept of the research is to use multiresolution techniques and hierarchical clustering/partitioning methods to make large dataset analysis more manageable.

Version 4.0, developed by Ying-Huey Fua at WPI, represented a major change from numerous perspectives, not the least of which was that it no longer depended on the X-Window environment. Using OpenGL and Tcl/tk, the package would now run on both Unix and Windows platforms. A new interface, featuring Windows-like menus and icons to represent the different display modes, has a very different look-and-feel from previous versions. This version also introduced a new method for exploring large multivariate data sets, based on hierarchically clustering the data and presenting aggregation information rather than just the raw data. Hierarchical parallel coordinates, described in a paper at Visualization '99, show cluster summarizations via shaded bands rather than single polylines, conveying both cluster extents and size. Bands are colored via their location in the hierarchy, such that child and parent clusters have similar clusters. A new brushing paradigm, called structure-based brushing, was also introduced. This allows users to navigate the hierarchy, selecting subtrees for which they would like to display more or less detail. This new technique for interaction was reported in a paper at Information Visualization '99, and also in an extended paper in IEEE Transactions on Visualization and Computer Graphics.

The new millenium: 2000-2004

The XmdvTool version 4.1 was released on October 4, 2000. It extends the support for hierarchically structured data to scatterplot matrices, star glyphs, and dimensional stacking. Using the same coloring and navigation strategies pioneered in Version 4.0, Jing Yang has created a consistent and complete strategy for large-scale multivariate data exploration. She has also eradicated several bugs found in version 4.0, such as bands extending beyond the drawing area boundaries, and made many improvements in the efficiency of the code.

The XmdvTool version 4.2 was released on October 1, 2001. It finally tied together the visualization front-end with a database backend. Using a novel indexing structure for hierarchies, Daniel Stroe has developed efficient query mechanisms for all navigation operations on the structure-based brush. He has also experimented with caching and prefetching techniques to further improve performance. All of these additions have been integrated into an Oracle backend which communicates seamlessly with the existing visualization front-end. A new glyph placement dialog is added to XmdvTool in this version. You could now place glyphs according to the order of one of their dimensions, place them in a 2D space composed of two of their dimenisons, or even place them using their principal components. You could even space the glyphs or deduce the overlap to make the pictures look better! The glyph placement is now limited to the flat displays.

XmdvTool Test Version 5.0 was released on July 12, 2002 and Beta Version 5.0 was Released on March 20, 2003. The 5.0 version improves upon XmdvTool 4.2 in the following aspects:

  • Database support: Database connectivity to Oracle 8i added to allow for more efficient and faster access to existing, larger databases.
  • User interface: New and improved user interface. Includes a Quick Menu to speed up your operations, plus many more toolboxes to assist in data navigation and tool navigation.
  • Data investigation: Distortion capability added to all displays which allows separate zooming of horizontal and vertical axes for more detailed investigations of interesting data zones. Dimension selection/de-selection and re-ordering also added to allow customized selection of interesting dimensions.
  • How-To support: Help menu expanded to guide you through the extended set of features.

XmdvTool 6.0 Alpha Version was released on September 1, 2003. It improves upon XmdvTool 5.0 in the following aspects:

  • Dimension Reduction (Tools->Dimension Reduction): Visual Hierarchical Dimension Reduction (VHDR) added to allow for more efficient and effective exploration of high dimensional dataset. InterRing is used as the interface of VHDR.
  • Time Series Animation (Tools->Time Series Animation): It is added to allow you to set a dimension as the "time" dimension and watch how the data items appear and disappear according to their value of the "time" dimension.
  • Color Scale and Color Mapping: Six additional color scales (lightness, saturation, lightness + saturation, grey scale, magenta and heated object) are added into system for better color presentation. The different dimension values are used for structure driven color mapping.
  • Performance: Some users have complained about the speed of XmdvTool 5.0. We have removed several "bugs" that slow down the system so that XmdvTool 6.0 is much faster than the 5.0 version.
  • Xmdv Meta File Support: A new file format for description of the types of dimensions and mapping between the values and numeric representations has been added. This allows to include nominal variables, document types and various user-defined complex types to be included and displayed within Xmdv. This is still an experimental feature and currently compiles on Windows/VC6 using the USE_META definition. Use the website for more details on this feature and the meta file format.

XmdvTool Light Java Version was released on May 25, 2004. It is a light-weight version of XmdvTool. It contains the four major multi-dimensional displays of XmdvTool: parallel coordinates, star glyphs, scatterplots, and dimensional stacking. It also supports the basic brushing: users can manipulate a brush to highlight a subset of data items.

The new beginning: 2006 - current date
The new directions we are looking at are (1.) Quality Space Visualization, (2.) Managing Discoveries in Visual Analytics, (3.) Interactive Stream Views: Visual Analysis of Streaming Data, and (4.) Model-Driven Visual Analytics on Streams.

XmdvTool 7.0 version was released on October 26, 2006. It improves upon XmdvTool 6.0 in the following aspects:

  • Visualization using pixel-oriented techniques, including linked brushing between all visualizations, powerful data reordering capabilities, and dimension placement based on multidimensional scaling.
  • Enhanced handling of nominal data, including using correspondence analysis to do data-driven conversion of nominal to ordinal values.
  • Automated visual clutter measurement and clutter-based dimension reordering.
  • Interactive control of dimension binning for dimensional stacking.
  • Support for HDF4 and HDF5 data formats, as well as Oracle databases, Excel spreadsheets, and XmdvTool's native .okc format.

XmdvTool 8.0 version was released on October 20, 2010. Starting from XmdvTool 8.0, we use SourceForge to manage all releases. Please go to XmdvTool sourceforge link to download XmdvTool 8.0 Windows and Linux/Unix versions. The salient features of XmdvTool 8.0 are as follows:

  • New software architecture: The new system is based on the information visualization reference model (or visualization pipeline) developed by Ed Chi. For more details on our extensions to this pipeline, please see our paper in VDA 2010.
  • New development environment: We have ported XmdvTool to Eclipse using Qt for the UI to enhance portability.
  • Multiple views: User can open multiple datasets at once, and observe each dataset in multiple sub windows with different visualizations. These windows can be tiled and/or cascaded.
  • Color strategy: With a new color strategy dialog, users can assign colors to datapoints based on data values or different orderings. We support sequential, diverging, and qualitative color maps based on Cynthia Brewer's work.
  • CSV file support: We enable users to open comma-separated values (CSV) files directly in XmdvTool, in addition to the XmdvTool native file format (.okc).

Beyond that, who knows? We have lots of ideas for future directions, including ways to deal with large numbers of dimensions, user-guided reclustering, non-numeric data, and a host of other suggestions made by current users. We hope to have a major release every 18 months or so, with minor releases as warranted.

Acknowledgement

Over the years the XmdvTool project has been supported by NSF under grants IIS-9732897, IRIS 97-29878, IIS-0119276, IIS-0414380, IIS-0812027, and CCF-0811510. The latest awards on Managing Discoveries in Visual Analytics and Interactive Stream Views: Visual Analysis of Streaming Data can be found at IIS-0812027 and CCF-0811510, respectively.