Traditional sensing applications have been limited to the collection of environmental data such as weather conditions or pollution levels. The availability of personal ubiquitous technologies, such as smart mobile phones and personal digital assistants (PDAs) enables a much richer set of applications and systems, not only restricted to sensing information about inanimate objects but also about people. They can be used to collect information about the individuals such as their current position, their activities inferred by means of the sensors embedded in the phones, the presence of other people or objects around them as well as how they interact with other people. The researchers at Cambridge
Dr Mirco Musolesi, Dr Cecilia Mascolo and Dr Jason Rentfrow at the CamBridgeSens are working hard to try to infer people emotions such as anger, animosity, and calmness during a conversation by combining the various raw data from the sensors such as voice pitch and volume and fast movements
Their research challenges are of twofold: they are looking at it from an algorithmic perspective, developing efficient inference algorithms that are able to combine the sensor raw data in order to extract high-level information such as user activity and roles; and secondly from an engineering perspective were they will have to build an efficient system for this class of resource-constrained devices considering the power consumption.
Below is the interview with the researchers.
Describe your project?
Psychology would like to predict behaviour based on data. How will a person behave based on their personality? It’s hard to collect personality data in a psychological experiment/lab. How can we collect behavioural data in the “real world”? Experience sampling started with pagers, but now mobile phones are used. People are paged periodically during the study and asked to complete a brief survey. This is more reliable than a behavioural check list, but people may not tell the truth. Using sensors to track people is a good new way to follow people unobtrusively.
We have:
- Used Bluetooth to see if two people are in the same room
- Used magnetometers to understand the mutual position of the people (and found that the data quality is lacking with the current technology)
- Used accelerometers to see if people are moving
- Used microphones to discover patters in conversation, (whether they are in traffic, outside, watching television)
- Developed the communication component involving an expert in speech recognition systems
- All data is collected and sent across the Internet to a database
- Tried to create rules to create automatic analysis of data – eg ‘if’ Jason and Mirco are collocated for more than 10 min, they are friends
- Adapted the technology – for example by getting phones to probe (e.g. test for 1 sec, is someone speaking? If yes, record, if not, turn off and probe again an 1 minute)
- Completed a GPS component
- Learned lessons – limitations of sensors, what is feasible to measure, the need to develop efficient algorithms for data mining, investigated what do we do about privacy issues
Where is your project going next?
20 phone deployment, voice recognition, small test with fewer people in psychology, make inferences about interaction with other people, differentiate between interactions between friends, acquaintances, strangers; invite people in and video tape actions and match with sensor data to see if it’s accurate.
How has the Innovation Competition prize benefited you?
Got us to interact. Provided the motivation to reach out to the Psychology Department. Now we know the level on which we can rely on phones for a proposal and will not promise something we can’t deliver. The purchase of the phones was invaluable; it allowed us to begin the pilot immediately and we now have the phones for future work. This was a great addition to the lab.
Are there any useful directions you see CamBridgeSens going in for the future?
It would be good to see more social science people involved. Faculty could be useful in pointing out how they could use the phones to study things like bullying, parenting, etc.
Source