One, two, three, many!

I published recently a paper in the 2018 European Conference on Ambient Intelligence about counting the number of persons in a Smart Home, and I thought trying to popularize it would be a good first article for this blog. The paper is called Online Guest Detection in a Smart Home using Pervasive Sensors and Probabilistic Reasoning

A Smart-what? Perva-what?

Well, I am quite certain most of the readers of these lines (if not all) have a good idea of what a Smart-Home is, but pervasive sensors might be more obscure. Let’s make sure everyone is on the same page and set up some things straight.

A Smart-Home is a home (house or apartment generally) equipped with various Pervasive Sensors. Pervasive Sensors is a term referring to very simple wireless sensors, of the place-it-and-forget-it type. They are pressure sensors, motion sensors, humidity sensors, light sensors… You get the idea: they are sensors that measure a lot of features from the home. Taken individually, these sensors just give one indication and can be used for automation, such as turning on your light bulb automatically when the light goes low, or increasing the heating in your home if it’s too cold. However, it is possible to combine all these pieces of information to infer higher-level information, such as the fact that you are taking a shower or sleeping… Scary for privacy? Could be, but we’ll talk about this later. Also don’t forget that the same technology can detect when you fell and call for help, or offer you personalized services depending on your activity. This is especially good for elderly persons and persons with disability.

In a Smart Home, you can have more than Pervasive Sensors: you can have cameras (an alarm system for instance) or microphones (in a device such as Alexa or Google Home). Those are not considered in our work: we based our study exclusively on Pervasive Sensors.

But why do I need to count?

A lot of applications currently being developed use these sensors to perform activity and context recognition. Activity recognition is, obviously, recognizing what you are doing (sleeping, eating, doing sport…). Context recognition is slightly higher level as it uses activity recognition and more information (from the sensors for instance, but not only) to recognize situations, such as a risk of fire (if you are sleeping but your stove stayed on) or a low physical activity (if you haven’t moved from your bed for some days. This is especially important for elderly).

However, most of these systems are based on one assumption: they know how many persons are in the home at each instant. Could be 1 or 2 (rarely more with current systems),  but we know. If you are receiving guests, the system does not know anymore who is activating the sensors and cannot work anymore. This is why knowing how many persons are in the environment (a.k.a. counting) is important.

Ok, got it. So how do you do that?

To count the number of persons in a Smart Home, we based our work on three assumptions1.

The first assumption is that a ubiquity is a very rare gift and one person cannot be at two places at once2. Therefore, if two sensors activates at the same time in two different places, therefore it means that two different persons are there. Really? Well, almost. Have a look at this layout of a flat and the sensors in it:What do you think will happen if someone sits on a chair in the living room and starts playing some board game? Bingo: the distance sensor on the chair and the motion sensor of the living room are probably going to be activated at the same time. In this case, we say that both sensors are connected.Ok, so let’s change a bit: since our test subject don’t have the gift of ubiquity, if two disconnected sensors are activated at the same time, it means that two persons are in the home. Yep, you got it! That’s exactly it!Now the problem is that errors in the sensor detection can happen (sometimes the sensor will activate for nothing or won’t activate when it should). So to deal with these problems, we are using Probabilistic Reasoning, and more precisely HMM (Hidden Markov Models).I am going to make a bigger article on HMM later, but for now let’s say that they are structure specialized in reasoning and detecting temporal patterns. That’s why they are very useful here: they can use previously recorded information to infer something at the current time. And this is where our two other hypotheses is coming in3. Our second hypothesis is that the gift of teleportation is as rare as the one of ubiquity4. The last hypothesis is that there is only one entry point in the environment, meaning only one place from which individual can enter and leave the home. It is a reasonable hypothesis for apartment where you usually have only one entrance door, but it becomes more constraining for houses. That’s one of the limitation of our work. So, to make things clearer: let’s say that at a timestamp t1, the motion sensor in the living room and the motion sensor in the bathroom are activated. According to our first hypothesis, there is two persons in the environment (because the two sensors are disconnected). Then, at the next timestamp t2, the motion sensor in the living room is the only one activated. Since there is no way that someone could have exited the apartment or teleported from the bathroom to the living room without being detecting by another sensor in the entrance, this is called an inconsistency and we consider that there is still two persons in the apartment even if only one sensor is activated. However, if for a certain amount of time after there is still one sensor activated, then we consider that we missed the user existing the environment and that there is finally only one person.

On the next figure, you can see the overall architecture of our framework:Don’t worry, it’s just what I have been explaining before. So the first part of our framework takes the sensor in the environment and creates a co-activation graph (i.e. the connected/disconnected relations we mentioned earlier). Then this, graph is translated into a set of constraints that are used by a reasoning module called a Constraint Satisfaction Problem Solver. In brief, it is taking a set of constraints and some sensor data (i.e. which sensors are activated) and try to find a solution (here, how many persons in the environment) that satisfies all the constraints. It generates a series of observations (here, how many persons it estimates in the environment) that is used by the Probabilistic Reasoner (the HMM) to resolve inconsistencies.

And this works?

It works quite well. For instance, we are capable to detect between 1 person and more with an accuracy of almost 95%. It’s already quite useful to detect if the person receives guests. We can also discriminate really well between 1, 2 and more. And we can count with a reasonable accuracy until 4 persons. Afterwards, it drops quite a lot.

So there is room for improvement, of course, but since this is, to our knowledge5, the first paper capable of counting with only pervasive sensors, it’s not that bad!

So what is there to improve?

Well, as I already said, there is the assumption of only one entry point which is quite strong if we consider houses and we should work to remove it. Then obviously we need to study the impact of the distribution of sensors in the environment in order to see how it affects the performance of our system. Indeed, we already noticed through experiments that the performances depends greatly on where you place your sensors and how many there are, but it would be nice to have more information on this topic. And finally,  finding a way to make it more precise of course. Even if for now, I have no idea on how to do it…

Paper please?

The paper is fully available here! Enjoy!

  1. Get used to this sentence… Most of AI-related scientific papers are based on assumptions and it is very important to make them clear.
  2. And for those of you who are lucky enough to have the gift of ubiquity, please be kind not to use it while testing our system
  3. I hope you didn’t think I forgot then!
  4. Once again, those lucky enough to have the gift of teleportation, please do not use it while testing our system
  5. Get used to this sentence too. This is a classic in papers. Since you can never say with certainty that you have reviewed the whole litterature on the subject, you always need to be careful and always make this clear that yours claims holds as long as you haven’t missed something
0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.