Merging Interactive Visualization and Automated Analysis
for Group Decision Making Involving Large Datasets

Evanthia Dimara, Anastasia Bezerianos and Pierre Dragicevic
2014

Abstract: Many critical human decisions involve large datasets and are made by groups of people with different backgrounds. Besides a general lack of understanding of the dynamics of group decision making, there is a lack of effective and generic tools for supporting such decisions. The purpose of this PhD thesis will be to facilitate group decision making involving large datasets by investigating new tools that combine the strengths of automated analysis with the strengths of visualization. Such tools will follow the model of mixed-initiative systems, where computers and humans collaborate closely in problem-solving. In addition, they will support group decisions by aggregating human input from various domains of expertise and mediating possible conflicts. Their design will be informed by problems from specific application domains and by the needs and capabilities of end users (either domain experts or the general public), while the underlying concepts will be expressed in a way that makes them easily generalizable to other domains and problems. The tools will be empirically evaluated according to their ability to support rational decision making. Overall this thesis will contribute to the advancement of scientific knowledge and society by facilitating evidence-based decision making in a wide array of disciplines.

Résumé: Un grand nombre de décisions importantes se font sur des grands jeux de données et en groupe. Ces situations sont susceptibles de produire des décisions sous-optimales ou irrationnelles, et il existe à ce jour peu d’outils informatiques capables de mitiger ce problème. Le but de cette thèse est de faciliter les décisions de groupe portant sur des grands jeux de données en explorant de nouveaux outils qui combinent les avantages de l’analyse automatique avec les avantages de la visualisation d’information. Ces outils seront calqués sur le modèle des systèmes à initiative mixte, où ordinateurs et humains collaborent de façon proche pour résoudre des problèmes difficiles. En outre, ces outils devront faciliter les décisions de groupe en agrégeant les opinions issues d’individus d’expertises différentes, et en jouant de rôle de médiateur en cas de conflits. La conception de ces outils s’inspirera de problèmes réels dans des domaines d’applications choisis par le candidat, ainsi que des besoins et des capacités des utilisateurs finaux (experts ou grand public). Les concepts sous-jacents seront cependant exprimés d’une manière qui les rend facilement généralisables à d’autres domaines et problèmes. Enfin, ces outils seront évalués de manière empirique en fonction de leur capacité à améliorer la qualité des décisions. En résumé, cette thèse devrait contribuer à l’avancement à la fois des connaissances scientifiques et de la société en facilitant la prise de décision rationnelle dans une variété de disciplines.

Extended Abstract

Many human decisions involve large datasets and are taken by groups of people with widely different expertises. Such decision-making processes abound in domains such as social network analysis (Bezerianos et al, 2010), biology research (Regan et al , 2005) and business intelligence. They are also involved in a number of critical domains like civil crisis management (Dragicevic et al, 2008), homeland security, foreign affairs, environmental sustainability, jury trials (Koehler, 1997), and pharmaceutical drug regulation (Welch and Black, 2010). These domains often require complex human decisions with possibly human lives at stake, such as: "how many fire brigades should we send to this area?", "what sort of financial support should we provide to this country?", "do we have enough scientific evidence to approve this drug?" or "should we sentence this individual?".

Studies on cognitive biases and human judgment have already demonstrated that the process of decision making is suboptimal for single individuals facing extremely simple situations (Plous, 1993, Kahneman, 2011). Decisions involving large datasets are likely to be even more challenging and subject to costly errors, especially since large datasets escape the understanding of even domain experts and scientists. Although collaboration has been shown to yield better decisions in some cases (Surowiecki, 2005), involving groups adds another level of complexity to the problem: group decisions can involve large numbers of people (Arrow, 1950), participants may have conflicting interests (Camerer, 2003, Panzarasa et al, 2001), and some people may exert a detrimental influence (Asch, 1951). These different issues have already been widely studied (e.g., in economics, game theory, voting theory, ethical philosophy and social psychology), though usually not in combination with visual analytics tasks and settings.

Besides a general lack of understanding of the dynamics of group decision making involving large datasets, there is a lack of effective and generic tools for supporting such decisions. One problem is that traditionally, computer-assisted decision making is split into two antagonistic approaches: automated analysis and information visualization.

Automated analysis tools range from simple statistical procedures to expert systems involving machine learning and artificial intelligence. Although these approaches are reaching maturity, they are not sufficient to support complex decisions for several reasons. First, they ignore expert knowledge that cannot be formalized (Chilana et al, 2010, Tory et al, 2009, Wegner, 1997). Then, analysis tools often act as "black box" who inner workings are hard to understand, even sometimes by experts (Gersh et al, 2005). In other cases, analysis tools restrict users by expecting a very particular input, ignoring other context-relevant information that the users may have (Teevan et al, 2004). Thus automatic analysis is powerful but often results in tools that are difficult to control and predict, and leave little room for subjective judgment.

In contrast with automatic analysis, information visualization delegates most of the thinking to humans by capitalizing on visual perception (Fekete et al, 2008, Card et al, 1999). But it falls short in supporting effective decision making: exploratory data analysis tools are useful for generating hypotheses, but they often hide uncertainty in the data (Streit et al, 2008, Amar and Stasko, 2005) and do not shield decision makers against perceptual and cognitive biases (Hulman, 2011, Plous, 1993, Kahneman, 2011). Visualization tools are currently designed and evaluated based on data retrieval and insight tasks, rather than the ultimate task of decision making (Amar and Stasko, 2005).

The purpose of this PhD thesis will be to facilitate group decision making involving large datasets by investigating new tools that combine the strengths of automated analysis with the strengths of visualization. We believe that sensemaking of large datasets and complex decision making is the most effective when assisted with machine learning, while remaining under the control of end users and informed by their expertise (Chau et al, 2011). Our tools will focus collaboration between users (Boland et al, 1992) and between users and computers according to a mixed-initiative paradigm (Shieber, 1996). Computer tools will also facilitate collaboration and mediation between users (Applegate et al, 1986, Kittur, 2010)

In this thesis, interactive visualization will be explored for its potential to i) inform users by showing data and facts together with their strength of evidence, ii) increase the transparency of analysis methods by revealing how they operate and making them tunable, iii) support group communication, in particular through co-located and remote multi-surface environments (Beaudouin-Lafon and al., 2012).

Automated analysis will be explored for its potential to i) simplify large data sets by hiding details irrelevant to the decisions to be made, ii) help users make rational decisions by taking into account uncertainty in the data, prior probabilities, and human biases, iii) optimize group thinking by aggregating human input from various domains of expertise and mediating conflicts. The system will be designed to be egalitarian: its internal logics should be transparent, controllable, and approved by every group member.

This topic is kept purposefully broad so that the student will be free to focus on subtopics that match his/her educational background, skills and personal interests. The design of the tools will be informed by problems from specific application domains and by the needs and capabilities of end users (either domain experts or the general public), while the underlying concepts will be expressed in a way that makes them easily generalizable to other domains and problems. The tools will be evaluated according to their ability to support rational decision making using well-established empirical methods (e.g., from behavioral economics).

The thesis advisors have an extensive expertise and experience in the key fields related to this topic: Pierre Dragicevic has been doing research in human-computer interaction and information visualization for 15 years and his research is orienting towards the study of information visualization for decision making (Micallef et al, 2012). Anastasia Bezerianos will also be a key collaborator due to her expertise in computer-supported collaborative work in human-computer interfaces and information visualization systems (Bezerianos, 2008, Isenberg, 2009). The two co-advisors have a long and successful experience of collaboration and coauthored several major papers together. An initial extended draft of this proposal has been written by Evanthia Dimara, our PhD candidate, and therefore it closely reflects her research interests.

Overall this thesis will contribute not only to the advancement of scientific knowledge, but also to the advancement of society by facilitating evidence-based group decision making in a wide array of disciplines.

References

  • Amar R. A. , Stasko J. T. , Knowledge Precepts for Design and Evaluation of Information Visualization, IEEE Transactions on Visualization and Computer Graphics - TVCG , vol. 11, no. 4, pp. 432-442, 2005
  • Applegate L. M. , Konsynski B. R. , Nunamaker J. F. A group decision support system for idea generation and issue analysis in organization planning. Conference on Computer Supported Cooperative Work - CSCW , pp. 16-34, 1986
  • Arrow, K. J. (1950). A difficulty in the concept of social welfare. The Journal of Political Economy, 328-346.
  • Asch, S. E. (1951). Effects of group pressure upon the modification and distortion of judgments. Groups, Leadership, and Men. S, 222-236.
  • Beaudouin-Lafon, M., Huot, S., Nancel, M., Mackay, W., Pietriga, E., Primet, R., ... & Klokmose, C. (2012). Multisurface interaction in the WILD room. Computer, 45(4), 48-56.
  • Bezerianos, A., Chevalier, F., Dragicevic, P., Elmqvist, N., & Fekete, J. D. (2010). Graphdice: A system for exploring multivariate social networks. In Computer Graphics Forum (Vol. 29, No. 3, pp. 863-872). Blackwell Publishing Ltd.
  • Bezerianos, A., & McEwan, G. (2008). Presence disparity in mixed presence collaboration. In CHI'08 extended abstracts on Human Factors in Computing Systems (pp. 3285-3290). ACM.
  • Boland R. J., Maheshwari A. K. , Te'eni D. ,Schwartz D. G. , Tenkasi R. V. Sharing perspectives in distributed decision making. Conference on Computer Supported Cooperative Work - CSCW , pp. 306-313, 1992
  • Camerer, C. (2003). Behavioral game theory: Experiments in strategic interaction. Princeton University Press.
  • Card, S. K., Mackinlay, J. D., & Shneiderman, B. (Eds.). (1999). Readings in information visualization: using vision to think. Morgan Kaufmann.
  • Chau D. H. , Kittur A. , Hong J. I. , Faloutsos C. , Apolo: making sense of large network data by combining rich user interaction and machine learning : Computer Human Interaction - CHI , pp. 167-176, 2011
  • Chilana P.K. , Wobbrock J. O. , Ko A.J. Understanding usability practices in complex domains. Computer Human Interaction - CHI , pp. 2337-2346, 2010
  • Dragicevic P., Fekete J-D, Stromer J, Euphrosine J. (2008), The Taranis Project. http://www.aviz.fr/old/taranis/
  • Fekete, J. D., Van Wijk, J. J., Stasko, J. T., & North, C. (2008). The value of information visualization. In Information visualization (pp. 1-18). Springer Berlin Heidelberg.
  • Fekete, J.D. (2008) Analysis and Visualization. http://www.inria.fr/en/teams/aviz.
  • Gersh, J. R., McKneely, J. A., & Remington, R. W. Cognitive Engineering: Undertanding Human Interaction with Complex Systems. Johns Hopkins APL Technical Digest, 26, 4 (2005), 377-382.
  • Hullman J., Diakopoulos N., Visualization Rhetoric: Framing Effects in Narrative Visualization, IEEE Transactions on Visualization and Computer Graphics - TVCG , vol. 17, no. 12, pp. 2231-2240, 2011
  • Isenberg, P., Carpendale, S., Bezerianos, A., Henry, N., & Fekete, J. (2009). Coconuttrix: Collaborative retrofitting for information visualization. Computer Graphics and Applications, IEEE, 29(5), 44-57.
  • Kahneman, D. (2011). Thinking, fast and slow. Macmillan.
  • Kittur A. , Kraut R. E. Beyond Wikipedia: coordination and conflict in online production groups. Conference on Computer Supported Cooperative Work - CSCW pp. 215-224, 2010
  • Koehler J. J. . One in Millions, Billions and Trillions: Lessons from People V. Collins (1968) for People V. Simpson (1995). Legal Education, 47:214–223, 1997.
  • Micallef, L., Dragicevic, P., & Fekete, J. (2012). Assessing the effect of visualizations on bayesian reasoning through crowdsourcing. Visualization and Computer Graphics, IEEE Transactions on, 18(12), 2536-2545.
  • Panzarasa P., Jennings N. R. ,Norman T.J., Formalizing Collaborative Decision-making and Practical Reasoning in Multi-agent Systems, Journal of Logic and Computation - LOGCOM , 2001
  • Plous, S. (1993). The psychology of judgment and decision making. Mcgraw-Hill Book Company.
  • Regan H. M. , Ben-Haim Y. , Langford B., Wilson W. G. , Lundberg P., Andelman S. J., Mark A. Burgman Robust Decision-making Under Severe Uncertainty For Conservation Management Ecological Applications - ECOL APPL , vol. 15, no. 4, pp. 1471-1477, 2005
  • Shieber, S. M. A call for collaborative interfaces. ACM Computing Surveys (CSUR), 28, 4 (1996)
  • Streit A, Pham B. ,Brown R. A Spreadsheet Approach to Facilitate Visualization of Uncertainty in Information IEEE Transactions on Visualization and Computer Graphics - TVCG , vol. 14, no. 1, pp. 61-72, 2008
  • Surowiecki, J. (2005). The wisdom of crowds. Random House LLC.
  • Teevan J., Alvarado C., Ackerman M. S. , Karger D. R. The Perfect Search Engine Is Not Enough: A Study of Orienteering Behavior in Directed Search Computer Human Interaction - CHI , pp. 415-422, 2004
  • Tory M. , Möller T. Human Factors in Visualization Research, IEEE Transactions on Visualization and Computer Graphics - TVCG , vol. 15, no. 6, pp. 1073-1080, 2009
  • Wegner, P. (1997). Why interaction is more powerful than algorithms.Communications of the ACM, 40(5), 80-91.
  • Welch H. G. and Black W. C. Overdiagnosis in Cancer. J National Cancer Institute, 102(9):605–613, 2010.