Just-In-Time Information Retrieval and Summarization for Personal Assistance

Staff - Faculty of Informatics

Date: / -

USI Lugano Campus, room SI-003, Informatics building (Via G. Buffi 13)

You are cordially invited to attend the PhD Dissertation Defense of Seyed Ali Bahreinian on Tuesday June 18th, 2019 at 15:30 in room SI-003 (Informatics building)

With the rapid development of means and channels for producing user-generated data opportunities for collecting such data over a time-line and utilizing it for various human-aid applications are more than ever. Wearable and mobile data capture devices as well as many online data channels such as search engines are all examples of means of user data generation. Such user data could be utilized to model user behavior, identify and retrieve information relevant to a user, and for personal assistance. Such user generated data can include recordings of one's conversations, various signals such as images, biophysical data, health-related data captured by wearable devices, interactions with smartphones and computers, and more. In order to utilize such data for personal assistance, its viable to replay summaries produced from them to users in order to augment a user's memory, send notifications about important events to the user, predict the user's near-future information needs and retrieve relevant content even before the user asks. In this PhD dissertation, we design a personal assistant with a focus on two main aspects: The first aspect is that a personal assistant should be able to summarize user data and present it to a user. To achieve this goal, we build a Social Interactions Log Analysis System (SILAS) that summarizes a person's conversations into event snippets consisting of spoken topics paired with images and other modalities of data captured by the person's wearable devices. Furthermore, we design a novel discrete Dynamic Topic Model (dDTM) capable of tracking the evolution of the intermittent spoken topics over time. Additionally, we present the first neural Customizable Abstractive Topic-based Summarization (CATS) model that produces summaries of textual documents including meeting transcripts in the form of natural language. The second aspect that a personal assistant should be capable of, is proactively addressing the user's information needs. For this purpose, we propose a family of just-in-time information retrieval models such as an evolutionary model named Kalman combination of Recency and Establishment (K2RE) that can anticipate a user's near-future information needs. Such information needs can include information for preparing a future meeting or near-future search queries of a user.

Dissertation Committee:

  • Prof. Fabio Crestani, Università della Svizzera italiana, Switzerland (Research Advisor)
  • Prof. Marc Langheinrich, Università della Svizzera italiana, Switzerland (Internal Member)
  • Prof. Fernando Pedone, Università della Svizzera italiana, Switzerland (Internal Member)
  • Prof. Cathal Gurring, Dublin City University, Ireland (External Member)
  • Prof. David Losada, University of Santiago de Compostela, Spain (External Member)