The Power of APIs

APIs are reshaping how companies do business. Once solely the domain of software engineers, they have grown to affect all levels of company.

What is an API?

An API is a set of clearly defined methods of communication between various software components. A good API not only makes it easy to get at your data but also combine it with additional data.

Why are APIs important?

Screen Shot 2017-08-24 at 10.42.35APIs are essential for interpretability. Deployed sensors and sensor networks will be part of the infrastructure of buildings for the foreseeable future and beyond, a trend that has gathered pace over recent years and is expected to continue for a long time to come. If the data only goes from the sensor into a black box, there is no built-in flexibility. Not only can you now see the visualizations clearly and easily but you can also integrate the data with the existing CAFM System. The same data can also be integrated into the existing systems already in place and familiar to the client whether meeting room booking systems or building security systems ensuring maximum accessibility and control

Benefits of an API:

  • Freedom from vendor lock-in – easy independent access to data
  • Better security – the more people who can see and test a set of code, the more likely any flaws will be caught and fixed quickly.
  • Customizability
  • Flexibility
  • Interoperability – our list of current integrations is always growing. Contact us to see if your system can be linked together
  • Auditability
  • Try before you buy – why not do a pilot of 5 devices before you rollout to a large scale deployment?

A closed platform can be damaging long-term and expensive. Your data becomes locked up and customers can find themselves very much at the mercy of the vendor’s changing vision, requirements, dictates, prices, priorities, and timetable.

More about the OpenSensors APIs

cropped-os_badge_logoOpenSensors API’s enable you to extract data by a variety of means; project, location, organizational departments, types of sensors, types of messages etc. We have integrated with many CAMF systems, meeting room booking systems, and building security systems. It is a constantly reviewed and expanding list with many other integrations planned on our road map. It is not only possible to build integrations with the visualization systems you already have in place but also use our canned visualizations. The data can also be seen within our systems, but it’s really targeted at enabling flexibility and multi-purpose use of the data.

OpenSensors has an open data mission

Hardware is expensive. Implementation can often be difficult to reduce the data as much as possible. Unless the data is used by multiple systems effectively you are going to have to have a different use for the same data within the building. With closed systems, the same sensors will have to be redeployed for each specific set of data required. This does not make sense to us. Our basic mission is to make these data sets easily accessible. Whether that’s private data or open data, we want you to be able to collect the information and make it immediately reusable and interoperable with all the different systems that you are already or will be using.

Our systems are similar to the electrical voltage standards that allow you to purchase any appliance and plug it in delivering power to your device. OpenSensors allow you to plug in the different devices as needed. Our API allows your applications to connect and extract the data when and how it is needed. The data is no longer locked away in a proprietary system holding you to the ransom of one vendor and puts you back firmly in control.

By harnessing the power of data aggregation across smart building ecosystem, every stakeholder wins. Over and over again, we’ve seen that enriching the experience of all parties (business – agent – customer) generates a higher degree of success and satisfaction

About OpenSensors

OpenSensors aggregates data from a variety of sensors for the next generation of smart Building Management Systems. Our dashboards help you make the most effective use of your office space. With experience in helping more than 100 companies to combine data from new workplace sensors seamlessly, interoperating it with existing and familiar mobile and desktop systems, we aim to give you a fully comprehensive overview.

The Good Sensor

On a daily basis our customers and community ask us to recommend a sensor provider to buy from, you should ping me on hello@opensensors.io if you want us to recommend your sensor. Often the requirement is vague, “I need an air quality sensor to put on my street for $100?” or “What sensors shall I use to understand my space usage?”. My process of assessment has grown more refined over time because if the sensors we recommend are unsuitable or unusable our company’s reputation is also on the line by association.

So we have come up with our own unscientific way to rate the quality of a sensor that should be applied simply. Most large scale sensor rollout projects of 1K or more often have these requirements as well. It’s possible that sensor providers that don’t rate highly using our criteria produce good sensors but getting the below right takes iteration and discipline in design and the likelihood is that the provider will a higher chance of being able to deliver.

Battery life If a sensor is battery powered, the typical expected life of battery should be clearly stated. Buyers will often want some explanation of what typical means for your sensor i.e. if it’s a PIR sensor have you calculated battery life based on being triggered once a day? The last thing your customers wants to do is invest in a lot of sensors, plus the cost of installation in order to find out that the battery life is only % of what they expected as it will still cost them a lot of money to rip them out and return them.

Bonus point for sensors that publish their battery status as standard so that the sensor owners can have some warning before changing.

Heartbeats

Sensors should tell people whether they are still alive or not periodically. Depending on your battery and connectivity constraints, this can vary, the important thing is that the buyer should not find out a bunch of devices are not working because they haven’t been heard from in days or weeks. Top tip; Heartbeats every 10-60 minutes when possible is sufficient, anymore and it ceases to be informative.

Installation and maintenance procedure

In non consumer environments, the people installing and maintaining sensors are often not the technical design firms or manufacturers. Does your device clearly tell people how to install it, do you have helper applications so that they don’t have to configure firmware? We are working on some solutions for this but more on this later; hint it’s all about enabling people to install sensors efficiently and a non technical installer being able to walk away knowing that the device has joined the network correctly. Does your sensor come with mounting and fittings?

Do people have to unscrew the casing to change batteries? Have you tested this with people and verified it?

 Data Quality

Quality in my definition means, is the data from your sensor easily understandable for someone that doesn’t know your domain. The reality is that often manufacturers pass on the analogue value of the particular sensor and that is too low of an abstraction for most people trying to read it. Battery voltage is a good example, during its life an AA battery will go from 1.5v to about 0.8v, but it follows a curve specific to the device and the battery. Understanding how this maps to a percentage or days of life is often complex. If it’s not possible to do much conversions or processing on your sensor or gateway, perhaps a handy explainer when people buy your device making them understand what the data means.

Support

Please state clear terms for warranties and return procedures to protect your consumers. Consumer protection should naturally apply.

Finally developing high quality hardware is hard, I am always amazed at the skill and dedication it takes when hardware designers and engineers take an idea and get it to manufacturing stage. We try to manage the community’s expectations on sensors they should buy vs the attitude of ‘just throw around cheap sensors’. It would be better in terms of environmental sustainability and user experience to get into the habit of doing more with less sensor density. For more on this, see Dr Boris Adryan’s excellent blog post

I have purposefully not mentioned security in this post as security assessments come with a lot of complexity, will aim to write up on this sometime soon.

Many Thanks to Toby Jaffey for editing.

Taking the Air at the Turk’s Head

5258374476_2ce1b88c66_b

OpenSensors are pioneers in open data and the internet of things, surfacing a wide range of data sets for open analysis. As an open data aggregator we deliver content over a common infrastructure; whether air quality or transport data, you only have to think about one integration point. Future cities need low data transaction costs for friction free operation, bridging technical gaps slow progress, so keeping the number of integration points low makes sense everybody.

Our journey starts here, as we build out our open data content expect to see more stories, more insight and hopefully some catalysts for positive change.

Before our first story, consider what will make open data and the Internet of things useful.

We must bridge the gap from data to information, allow consumers to abstract away the complexity of IoT to ask questions that makes sense to them.

Take data from the London Air Quality network (LAQN), the network is sparse so it’s improbable our need maps directly to a sensor. By coupling some simple python code with OpenSensors data we’ll mash some LAQN data together to get some insight about air quality in wapping.

In this story I’ll show how we can bridge the information gap with some simple code, yielding valuable insight along the way!

Chapter 1: OpenSensors.io Primer

First a quick primer on how data is structured in OpenSensors.io (for more detail check out our forum and glossary of terms)

  • Devices – Each connected ‘thing’ maps to a secured device, things map one-to-one to a device
  • Topics – Data is published by devices to topics, a topic is a URI and is the pointer to a stream of data
  • Organisations (orgs) – An organisation owns many topics and is the route of an orgs topic URI
  • Payloads – Payloads are the string content of messages sent to topic URI’s, typically JSON

Also check out our RESTful and streaming APIs on the website for more background and online examples.

Chapter 2: Putting JSON to Work

You can use the OpenSensors REST API to gather data for research, but it comes in chunks of JSON which isn’t great for data science. For convenience i wrapped up some common data sources for London into a python class. Since IoT data is rarely in a nice columnar form it’s valuable to build some simple functions to shape the data into something a bit more useful.

Chapter 3: Introducing the Turks Head

I’m fortunate to spend a lot of time in Wapping, in and around the community of the Turk’s Head Workspace and Cafe, but unfortunately we don’t have a local LAQN sensor. With a bit of data science and OpenSensors.io open data we can estimate what NO2 levels might be around the cafe and workspace.

A simple way to estimate NO2 is a weighted average of all the LAQN sensors, in this case we derive the weights from the distance between the sensor and our location. Since we want to overweight the closest sensors we can use an exponential decay to deflate towards zero for those far away.

For the Turks Head sensors in Aldgate, Southwark and Tower hamlets and the City are the closest and have the biggest impact on our estimate.

Chapter 4: Getting into the Data

With our air quality time series, and our weights we can dig into what our estimates for the Turks Head look like (NO2 * weight). Here’s the series for NO2 over the last 20 days, it looks like the peaks and troughs repeat, and the falling or rising trend is persistent in between.

Trend followers in finance use moving averages to identify trends, for example the MACD indicator (moving average convergence divergence). MACD uses the delta between a fast and slow moving average to identify rising or falling trends, we’ll do the same. For our purposes we’ll speed the averages up using a decay of 3 and 6 periods (LAQN data is hourly and we are resampling to give estimates on the hour).

What can we conclude from the charts for The Turks Head? From the left hand chart we can see the data is little noisy, with a flat line showing some missing or ‘stalled’ data. Looking at the 3 and 6 period decayed averages the data is smoother, with the faster average persistently trending ahead of the slower one.

Even with fast moving decays the averages cross only a couple of times a day, showing persistence when in trend. So using a simple trend indicator and the LAQN we can build a simple air barometer for the Turks Head.

Good 3 period exp average < 6 period average (green) Bad 3 period exp average > 6 period average (red)

This is helpful because, given a persistent trend state, where we have a ‘good’ air now, we’ll probably have ‘good’ air for the following hour.

Chapter 5: What’s the trend across London?

So we now have means of defining how NO2 levels at the Turk’s Head are trending, but is the trend state predictable over a 24 hour period?

Remember we define good or bad air quality trend as:

Good ‘fast’ average < ‘slow’ average = falling NO2 Bad ‘fast’ average > ‘slow’ average = rising NO2

If we aggregate data into hourly buckets we can visualise how much of the time, over the past 20 days, a sensor has been in a up trend (‘good’) for a given hour.

x = hour of the day y = percentage of bucket that is in a ‘good’ state

We can see that for each 1 hour bucket (24 in total) there is a city wide pattern; if we aggregate across the city (using the same measure, the percentage of sensors in up or down trend) we get an idea of how NO2 trends over a typical day.

Our right hand chart shows the percentage of ‘good’ versus ‘bad’ NO2 sensor states across London over the past 20 days (collected from about 80 sensors over 20 days)

Now this is a really simple analysis but it suggests the proportion of ‘good’ trends across London is high before 7am, and then falls away dramatically during the morning commute. No surprises there.

But the pattern isn’t symmetrical; after peaking around lunchtime, when only ~20% of the cities sensors having improving NO2, NO2 falls throughout the afternoon. From a behavioural standpoint this makes sense; there is a more concentrated morning commute relative to the evening. Most of us arrive at the workplace between say 8 and 9am, but in the evening we may go to the gym, we may go out for dinner, or just work late. The dispersion of our exits from the city is wider than when we enter.

Chapter 6 – PM versus NO2

So we have considered NO2 as our core measure, in part because there are more sensors in the LAQN delivering this data than particulates. But let’s consider particulates for a moment, LAQN deliver PM10 and PM2.5 measures, the definition can be found here.

Our temporal curves for particles differ from NO2 taking longer to disperse during the evening rush hour (remember we are measuring percentage of sensors in a ‘good’ state). As a measure of air quality NO2 builds up faster, and decays faster once peak traffic flows have completed, whereas particles linger only fading deep into the night (on average).

Closing Thoughts

In our data set, NO2 and PM measures differ in their average behaviour over a typical 24 hour period.

  • Behavioural interventions will need to consider whether particulates or N02 are the most impactful.
  • How can we communicate air quality to our citizens, and relate their personal needs to the measures most impactful on their lives?
  • Do we need additional sensors to create a more dense air quality resource? How can we allocate funds to optimally support network expansion and air quality services?
  • Knowing the characteristics of a sensor (location, calibration, situation (elevated, kerb side, A or B road) will improve estimates, how can we deliver this meta-data?

Plenty of food for thought…………..information

Notes and Resources

Our stories are quick and dirty demonstrators to promote innovation and should be treated as such. All data science and statistics should be used responsibly 🙂

All of the code supporting this can be found on github with data sourced from opensensors’ LAQN feed, and I use a postcode lookup to get long/lat locations for wapping. I’ve also taken some inspiration from https://github.com/e-dard/boris and https://github.com/tzano/OpenSensors.io-Py so thanks for their contribution!

http://www.londonair.org.uk/ https://www.opensensors.io/

The Path to Smart Buildings

Google ‘principles of good architectural design’ and you’ll get links to technology, to buildings and all manner of other services. But it’s hard to find principles of design for the tech services that facilitate smart buildings. Let’s remind ourselves what a smart building is with the help of sustainable tech forum; ‘The simple answer is that there’s automation involved somehow that makes managing and operating buildings more efficient’. So the need is well documented but we want to bridge to the ‘practice of designing and constructing buildings’, after all that’s what architecture is about.

OpenSensors hosted its first Smart Building Exchange (SBeX) event in September, and we are grateful to the panelists and attendees who made it such a success. Our goal was to bridge the gap between widely documented features of smart buildings and the tech that underpins it. Through our workshops we decomposed tenant needs and identified services to support them using the value proposition canvas. We borrow from lean product design principles since building operators need to rapidly innovate using processes inherited from startups. Mapping the pains and gains of users to the features and products of the tech stack revealed a common theme, data infrastructure. Data is the new commodity that new services will be built upon, some will be open and others private, but data will be the currency of the next generation smart building.

Take integrated facilities management (IFM) where data serves the desire to deliver better UX at a lower cost with fewer outages. IFM has pivoted from a set of siloed software services to a set of application services overlaid upon a horizontal data infrastructure. For example:

  • Data science services will develop to identify ‘rogue’ devices operating outside expected patterns, they will identify assets that need inspection or replacement and schedule maintenance works using time and cost optimisation routines.
  • Digital concierge services will use personal devices, location based technology and corporate data (calendar and HR data) to optimise both user experience and spacial allocation.

So can we identify a tech architecture to support this pivot from monolithic apps? Data services facilitated by a central messaging backbone allows the complexity of building services to be broken down and tackled one service at a time, lowering the risk failure and allowing agile iterations at a reduced cost. Take the pillars of data driven applications for IFM as identified by our workshop group; predictive/reactive alerting and tactical/strategic reporting, how might we go about servicing these needs? Consider how the path to smart buildings outlined below could help build an IFM product.

  • Build the value proposition founded on a clear vision of what your users want.
  • Identify the data that will drive your smart building product including open data
  • Identify the sensors needed to gather your data, they could be mobile devices or occupancy sensors
  • Identify connectivity from the sensor to your data infrastructure, this might be radio to IP connected gateways or directly onto the local network via POE (power over ethernet)
  • Structure your message payloads and commit to schemas to deliver repeatable processes for message parsing and routing within your building
  • Configure your events turning your data into information using rules based platforms for IoT such as node red
  • Build widgets and data services that can be bound together for dashboarding. By identifying common user needs across the enterprise we can operate a leaner system stack
  • Build user portals and dashboards using your common data services and components
  • Validate tenant user experience through surveys and modelling tenant behaviour using occupancy devices
  • Iterate to improve using data gathered throughout the building to deliver better products and experiences

OpenSensors has firmly backed Open Source and Open Data as the best way to yield value from the Internet of Things choosing to collaborate with the tech community to enable facilities managers to build higher order systems focused on their domain expertise. Please contact commercialteam@opensensors.io should you have a need for a smart building workshop or are ready to build your next generation smart building product.