|
| |
Tutorial Information: Information Filtering, Classification, and Extraction
Sunday, January 9, 2000, 8:30am -12:30pm
Presenter: Michael J. Pazzani
Tutorial Information: Intelligent User Interfaces,
An Introduction
Sunday, January 9, 2000, 2pm - 6pm
Presenter: Mark Maybury
Information Filtering, Classification, and Extraction
Sunday, January 9, 2000
8:30am -12:30pm
New Orleans, Louisiana, USA
Michael J. Pazzani
University of California, Irvine
The vast amount of information available on the Internet has given rise to a
number of agents for locating relevant, useful or interesting information for
a given individual. Such agents perform tasks such as prioritizing, filtering,
or sorting electronic mail; filtering news group articles and locating
interesting articles in unread newsgroups; "clipping" articles from on-line news services;
constructing queries for Internet search engines to find relevant information;
guiding a user to find relevant information on the World Wide Web; notifying a
user when a significant change occurs to a web site or when an item of
interest goes on sale. This tutorial focuses on the technology for
filtering, classifying and extracting information.
To perform such tasks, a profile of the user's interests must be created. In
this tutorial, we will focus on the learning and representation of user
profiles, the methods for collecting user feedback, and the representation of
information sources. This tutorial will review a variety the findings from
several decades of research on information retrieval focusing on approaches to
information filtering and classification. Next, machine learning approaches to
classification will be described including decision trees, nearest neighbor algorithms, Bayesian
classifiers and neural networks. We will discuss how they may be used to
learn user profiles, how user profiles may be visualized and how the results
of search can be visualized. We will discuss evaluation of individual system components
and user studies that evaluate entire systems.
The technology will be illustrated with examples from a variety of
information agents including LIRA, NewsWeeder, WebWatcher, WebDoggie,
Fab, WiseWire, SavvySearch, FAQFinder, InfoFinder, Letizia, firefly,
InfoFinder, Syskill & Webert, DICA and the Remembrance Agent.
Tutorial Outline
Recent applications of Information Retrieval and Machine Learning on the
World Wide Web Information Retrieval Task:
Document Retrieval
Document Representation
Queries
Similarity
Evaluation (Precision Recall)
Document Classification
Rocchios algorithm
Visualizing search results
Clustering/ Scatter Gather
User Studies
Information Theory, Machine Learning
Information-based approaches for term selection
Classification learning
Bayes
Decision trees
Nearest Neighbor
Perceptron
Neural Networks
Support Vector Machines
Putting it all together: Syskill & Webert
Recent Advances
Multimedia retrieval
Collaborative Filtering
Weighted majority and Infinite attribute models
WordNet; MeSH and MedLine
Learning from Labeled and Unlabeled data
Latent Semantic Analysis
Information Extraction
Systems and Evaluations
MeSHBrowse
Cat-a-cone
Tutorial Audience
The intended audience of this tutorial is practitioners and researchers
interested in issues involved with applying machine learning and information
retrieval algorithms to classification and ranking of information on the
Internet. There are no special prerequisites for this tutorial, although
a familiarity with introductory AI concepts such as classification and search,
and basic knowledge of mathematics and probability will be expected.
Interest in Tutorial Topic
With the increased usage and visibility of the Internet, there has been increased interest in artificial intelligence applications and research in
providing automated means to assist a user in locating relevant information. For example, at the most recent AAAI session on Intelligent Internet Agents was overflowing, while sessions on many traditional AI topics were sparsely
attended. This talk focuses on one key aspect of Intelligent Agents: the filtering and classification of information. It covers approaches from both
the Information Retrieval and Machine Learning Community and illustrates
these with a variety of fielded systems
Background of Tutorial Presenter
Michael Pazzani is a professor and department chair in Information and
Computer Science at the University of California, Irvine. He has been
active in Machine Learning research for the past decade with numerous
publications in IJCAI, AAAI, Autonomous Agents and the International
Machine Learning Conference. He has taught a variety of courses
including Introduction to Artificial Intelligence at the undergraduate
level (8 times), Natural Language Processing at the graduate level and
graduate seminars in Machine Learning and Information Retrieval.
A brief CV is included below.
| NAME |
POSITION/TITLE |
| MICHAEL J. PAZZANI |
Professor |
EDUCATION/TRAINING
| INSTITUTION AND LOCATION |
DEGREE |
YEAR(s) |
FIELD OF STUDY |
| University of California, LA |
Ph.D. |
1984-1988 |
Computer Science |
| University of Connecticut |
M.S. |
1979-1980 |
Computer Engineering |
| University of Connecticut |
B.S. |
1976-1980 |
Computer Engineering |
Professional Experience
1995-Present: Chair,
Department of Information and Computer Science,
University of California, Irvine
July 1997-Present: Professor of Information and Computer
Science,
University of California Irvine
1992-1997:
Associate Professor of Information and Computer Science,
University of California, Irvine
1988-1992:
Assistant Professor of Information and Computer Science,
University of California, Los Angeles
1984-1988:
Ph.D. Computer Science,
University of California, Los Angeles
1980-1984:
Member of Technical Staff, and Group Leader AI technology
group (1983-84), Mitre Corp, Bedford MA
Selected Publications:
Lathrop, R. H., Steffen, N. R., Raphael, M., Deeds-Rubin, S., Pazzani, M.
J., Cimoch, P. J., See, D. M., Tilles, J.G. (1998). Knowledge-based
Avoidance of Drug-Resistant HIV Mutants. (Innovative Application Award
winner), in Proc. Innovative Applications of Artificial Intelligence Conf.,
Madison, WI, USA, July 27-29, 1998.
Billsus, D. & Pazzani, M. (1998). Learning Collaborative Information
Filters. Proceedings of the International Conference on Machine Learning.
Morgan Kaufmann Publishers. Madison, Wisc.
Pazzani, M. (in press). A Framework for Collaborative, Content-Based and
Demographic Filtering. Artificial Intelligence Review.
Pazzani, M. (in press). Learning with Globally Predictive Tests. The First
International Conference on Discovery Science Fukuoka, Japan.
Merz, C. & Pazzani, M. (in press). A Principal Components Approach to
Combining Regression Estimates Machine Learning.
Pazzani, M., See, D., Shroeder, E., & Tilles, J. (1997). Application of an
Expert System in the Management of HIV-infected patients. Journal of AIDS
and Human Retrovirology. 15:356-362.
Pazzani, M., (1997) Comprehensible Knowledge Discovery: Gaining Insight from
Data. First Federal Data Mining Conference and Exposition. pg 73-82.
Washington, DC.
Billsus, Daniel & Pazzani, M. (1997). Learning Probabilistic User Models.
in Workshop Notes of "Machine Learning for User Modeling", Sixth
International
Conference on User Modeling, Chia Laguna, Sardinia.
Pazzani, M., See, D., Shroeder, E., & Tilles, J. (1997). Application of an
Expert System in the Management of HIV-infected patients. Journal of AIDS
and Human Retrovirology. 15:356-362.
Pazzani, M. (1997). Searching for dependencies in Bayesian classifiers.
Artificial Intelligence and Statistics IV, Lecture Notes in Statistics,
Springer-Verlag: New York.
Pazzani, M., Mani, S., & Shankle, W. R. (1997). Beyond concise and
colorful:
learning intelligible rules. Proceedings of the Third International
Conference on Knowledge Discovery and Data Mining, Newport Beach, CA. AAAI
Press, 235-238.
Pazzani M., & Billsus, D. (1997). Learning and Revising User Profiles: The
identification of interesting web sites. Machine Learning 27, 313-331.
Pazzani, M., Muramatsu J., & Billsus, D. (1996). Syskill & Webert:
Identifying interesting web sites. AAAI Spring Symposium. Stanford, CA.
Starr, B., Ackerman, M., & Pazzani, M. (1996). Do I Care? -- Tell Me
What's Changed on the Web. AAAI Spring Symposium. Stanford, CA.
M. Ackerman, D. Billsus, S. Gaffney, S. Hettich, G. Khoo, D. Kim, R.
Klefstad, C. Lowe, A. Ludeman, J. Muramatsu, K. Omori, M. Pazzani , D.
Semler, B. Starr, & P. Yap (1997). Learning Probabilistic User Profiles:
Applications to Finding Interesting Web Sites, Notifying Users of Relevant
Changes to Web Pages, and Locating Grant Opportunities. AI Magazine 18(2)
47-56.
Yamazaki, T., Pazzani, M., & Merz, C. (1996). Acquiring and updating
hierarchical knowledge for machine translation based on a clustering
technique. In Wermter, Riloff & Scheler (Eds.) Connectionist, Statistical,
and Symbolic Approaches to Learning for Natural Language Processing.
Billsus, D., & Pazzani, M. (1996). Revising user profiles: The search for
interesting Web sites. International Multi-Strategy Learning Conference.
Harpers Ferry, VA.
Starr, B., Ackerman, M., & Pazzani, M. (1996). "Do-I-Care: A
Collaborative
Web Agent." Proceedings of the ACM Conference on Human Factors in
Computing
Systems (CHI'96), April, 1996, pp. 273-274.
Brunk, C., & Pazzani, M. (1995). A Linguistically-Based Semantic Bias for
Theory Revision Proceedings of the 12th International Conference of Machine
Learning.
Pazzani, M., Nguyen, L., & Mantik, S. (1995). Learning from hotlists and
coldlists: Towards a WWW information filtering and seeking agent. In
Proceedings of the Seventh International Conference on Tools with Artificial
Intelligence.
Ali, K., Brunk, C., & Pazzani, M. (1994). On Learning Multiple
Descriptions
of a Concept. In Proceedings of the Sixth International Conference on Tools
with Artificial Intelligence. New Orleans, LA: IEEE Press.
Back to Top
Back to Registration Page
Intelligent User Interfaces
An Introduction
Sunday, January 9, 2000
2pm - 6pm
New Orleans, Louisiana, USA
Information Technology Center
The MITRE Corporation
202 Burlington Road
Bedford, MA 01730, USA
maybury@mitre.org
Intelligent user interfaces promise to improve interaction for all.
Drawing upon material from the recently completed Readings in
Intelligent User Interfaces (Maybury and Wahlster, 1998), this
tutorial will define terms, outline the history, describe key sub-fields, and exemplify and
demonstrate intelligent user interfaces
in action.
Keywords
Intelligent user interfaces, intelligent multimedia interpretation
and generation, user and discourse modeling, agent-based interfaces,
model-based interfaces.
INTRODUCTION
Intelligent user interfaces (IUI) are human-machine interfaces that
aim to improve the efficiency, effectiveness, and naturalness of
human-machine interaction by representing, reasoning, and acting on
models of the user, domain, task, discourse, and media (e.g.,
graphics, natural language, gesture). Intelligent user interfaces are
multifaceted, in purpose and nature, and include capabilities for
multimedia input analysis, multimedia presentation generation, and the
use of user, discourse and task models to personalize and enhance
interaction. An online tutorial is available at
http://www.mitre.org/resources/centers/it/maybury/iui99/index.htm.
Multimedia Input Analysis
Whereas traditional interfaces support sequential and unambiguous
input from devices such as keyboard and conventional pointing devices
(e.g., mouse, trackpad), intelligent multimodal interfaces relax these
constraints and typically incorporate a broader range of input devices
(e.g., spoken language, eye and head tracking, three dimensional
gesture). For example, they support asynchronous, ambiguous, and
inexact input by applying more sophisticated analysis of input. These
systems allow the resolution of multimedia references, for example
enabling the user to say "Put that there" while gesturing to a
map, by correlating eye and hand gestures with the deictic expressions
"that" and "there". Integrated input from multiple devices promises to
simultaneously enhance communication efficiency, effectiveness (e.g.,
speed and accuracy), and naturalness. Intelligent interfaces can also
detect and correct errors utilizing models of the media, user,
discourse, and task.
Multimedia Output Generation
Whereas traditional interfaces draw upon pre-programmed or canned
presentations (e.g., windows, menus, dialogue boxes), automated
interface and presentation generation addresses the ability of a
system to select content, apportion that content to various media
(e.g., typed or spoken language, graphics, gesture), and realize those
media in an integrated and coordinated fashion. Key multimedia
generation tasks include managing the communication (i.e., reasoning
about plans and intentions), selecting content to achieve given
communicative goals, designing the presentation, allocating and
coordinating information across media, realizing media, and laying
them out.
Model-based Interfaces
Given the complexity, associated skill level, and time required to
build interfaces, researchers have focused on creating user interface
design and development environments. User interface management systems
(UIMS), software development toolkits containing components such as
windows, menus, and dialogue boxes, were originally designed to
address this problems. While UIMS foster design consistency and
enhance programmer productivity via code reuse, unfortunately, they
frequently mix interface code with application code. In contrast,
model based interfaces, separate applications into (at least) four
layers: application actions, dialog control, style rules
(specifications of presentation and behavior), and style program layer
(primitive toolkit objects composed by style rules). In addition to
supporting more declarative development, these systems can draw upon
the above automated input analysis and output generation techniques.
In contrast to interface software repositories, model-based interface
development environments promise automated design critique, refinement
and implementation.
Interaction Management
Context has always been recognized as critical to the effectiveness of
interaction. Context comes in many forms, typically explicitly
represented in models of the user, discourse, task, and
situation. Computational techniques to acquire, represent, and exploit
context enable systems to track and react to interactive dialogue.
More principled models of interactive participants are essential to
enable such intelligent behavior as negotiation, tailored explanation,
and error detection and recovery among communication participants,
both human and machine.
Agent-based Interaction
Agents have increased in prominence in applications, including as
search agents, desktop support (e.g., Microsoft's Office Assistant),
collaborative filtering (e.g., shopping recommenders), and for
intelligent distributed computing. Agents may assist by decreasing
task complexity, bringing expertise to the user (in the form of expert
critiquing, task completion, coordination) or simply providing a more
natural environment with which to interact. Research in this area
includes the use of agents to express system and discourse status via
facial displays, multimodal communication between animated computer
agents, and standards and open architectures for building agent based
multimodal interfaces. Key research questions include: What can and
should an agent do? How they should do it? How, when, and why should
they interact with the user when doing it?
Evaluation
A final area addressed by the tutorial will be IUI evaluation.
Benchmarking, hypothesis testing, and repeatable experiments are
fundamental to any scientific endeavor. Community-based evaluation
using standard corpora and tasks have been applied in several areas
related to intelligent interfaces, including speech, information
extraction, and information retrieval, although relatively little
evaluation has been systematically performed on IUIs. Performed
objectively, precisely, and comprehensively, evaluation can benchmark,
chart progress, and enable comparison of relative strengths and
weaknesses of approaches. Evaluations can be either glass-box
(internal) and black-box evaluation (end-to-end). Criteria for
evaluation might include quantitative measures (e.g., time to perform
tasks, accuracy of tasks, percent of interassessor agreement) as well
as qualitative ones (e.g., user indication of utility, ease of use,
naturalness). Important dimensions of the problem include considering
human-human vs. human-computer communication, spoken vs. written
communication, unimodal versus multimodal communication, direct
vs. mediated communication. We will discuss a range of techniques
available to the scientist and engineer including wizard-of-oz
experiments, simulations, and instrumentation of live environments.
Summary
Effectively implemented and deployed, intelligent user interfaces
promise many benefits. These include: 7 More efficient interaction --
enabling more rapid task completion with less work. 7 More effective
interaction -- doing the right thing at the right time, tailoring the
content and form of the interaction to the context of the user, task,
dialogue 7 More natural interaction -- supporting spoken, written, and
gestural interaction, ideally as if interacting with a human
interlocutor.
TUTORIAL STRUCTURE
The tutorial introduces intelligent user interfaces using the
following outline:
* Interaction Management, including user and discourse models and adaptation
* Multimedia input analysis
* Multimedia output generation
* Agent-based interaction
* Evaluation of intelligent user interfaces
The tutorial will include animations and demonstrations.
INSTRUCTOR
Mark Maybury received his M.Phil. in Computer Speech and Language
Processing (1987) and his Ph.D. in Artificial Intelligence (1991) for
his dissertation, "Generating Multisentential Text using
Communicative Acts" at Cambridge University. He was awarded an
MBA from RPI in 1989. Mark has organized multiple international symposia,
given tutorials, and published over fifty technical and tutorial
articles in the area of language generation, multimedia presentation,
text summarization, and intelligent multimedia information retrieval.
Mark is editor of Intelligent Multimedia Interfaces (AAAI/MIT Press,
1993), Intelligent Multimedia Information Retrieval (AAAI/MIT Press,
1997) and co-editor of Readings on Intelligent User Interfaces (Morgan
Kaufmann Press, 1998), Advances in Text Summarization (MIT Press,
1999) and Readings in Knowledge Management (forthcoming). Mark is
Executive Director for of MITREUs Information Technology Division.
REFERENCES
4. Horvitz, Eric (1997) Compelling Intelligent User Interfaces: How
Much AI is Enough?, Position statement, In Moore, J.; Edmonds, E.; and
Puerta, A. (eds.) Proc. of International Conference on Intelligent
User Interfaces (IUI97), ACM, Orlando, Florida.
5. Sullivan, J. W., and Tyler, S. W. (eds) 1991. Intelligent User
Interfaces. Frontier Series. New York: ACM Press.
6. Shneiderman, B. (1997) Direct Manipulation for Comprehensible,
Predictable and Controllable User Interfaces, In Moore, J.; Edmonds,
E.; and Puerta, A. (eds.) Proc. of International Conference on
Intelligent User Interfaces (IUI97), ACM, Orlando, Florida.
Back to Top
|
