Project Haystack Enables A Future Where A Push Of A Button Can Turn Device Data Into True Intelligence

Listen to this article

Below is our recent interview with John Petze, Executive Director at Project Haystack:

Q: Why do we need Haystack? And Why now?

A: The quick answer is that Project Haystack enables a future where a push of a button can turn device data into true intelligence reducing the cost of intelligent buildings and enabling operational teams to better understand and improve the operation and performance of their facilities.

But let’s start with some background on the problem Haystack solves. The rapid growth in connected smart devices (equipment systems, sensors, IoT devices) is creating dramatic increases in the amount and type of data available. But it turns out that it’s one thing to have access to data; it’s another to make it useful and actionable. The primary barrier to easily using device data is knowing what it means.

The reason data semantics (a big pretentious word so let’s just say tagging going forward) is important is that device data are stored in many different formats, communicated via numerous protocols, have inconsistent non-standard naming conventions, and have very limited descriptors to enable us to understand meaning without direct human knowledge of the device producing the data. Ideally, we want data to be self-describing. Without that, a time-consuming manual effort is required before data can be used effectively to generate value.

Throughout our increasingly software driven world the challenge of how to give data meaning to enable software applications to work with data more effectively is a core challenge being addressed in every segment of the software industry. The concept of tagging is one of the most accepted approaches to address this challenge. Think about tagging your emails in Gmail or tagging your photos in your favorite photo app so you can find ones with specific meaning. Haystack provides a standardized way to tag device data so that we (and software applications) know what it means.

Recommended: Securrency Raises $17.6 Million Series A Funding Round Led By WisdomTree, Highly-Credible Asset Manager

Q: There are a few Haystack versions available, with latest being version 4.0. What has changed? How has Haystack evolved over the past few years?

A: The understanding of the need for semantic modeling of device and equipment data has matured significantly in the last decade and the requirements and techniques for applying semantic modeling to equipment data are advancing rapidly. Haystack 4 builds on the 8 years of experience by the worldwide community in applying Haystack across thousands of buildings worldwide, the input from practitioners in the community throughout that time, as well the collaborators that have participated in the activities of Haystack Working Groups.

Haystack 1 pioneered the concept of applying semantic modeling to equipment and device data using the simple approach of applying “tags” to items to define what they “meant”. Tags described things like units of measure, as well as facts and characteristics about data. For example, the tags:

discharge, air, temp, sensor, point, unit:”°F”

tells us that a number represents a numerical value of discharge air temperature expressed in degrees F produced by a sensor. Without the simple tags mentioned above you couldn’t do much with the data coming from the sensor – you wouldn’t know what it meant. Haystack 1 therefore provided us with a standardized vocabulary to markup “things” and the data they produced. That provided the industry with its first widely adopted solution for standardized, open, data modeling for device and equipment data, which allowed us to agree on the terms to use to help define what things are. In the world of semantics that’s called a vocabulary.

Haystack 2 introduced a REST API, in 2013, to provide a standard way to query a system that applied the Haystack semantic model to its data. As the demand for open protocols and open systems in the built environment continued to rise, offering an open API was important to ensure customers had a standard way to easily access the data in their systems.

Haystack 3, released in 2016, added several new data types to help machines better understand and process the different types of data formats for the IoT. The importance of data types for machines can be thought of by using a simple example. Imagine getting added to an email chain, where some of the older messages were in a language you didn’t understand. You could copy and paste the text into an online translation tool and may be able to figure out what was said, but not as quickly or as easily, if you natively spoke that language. Every machine uses the concept of a String. What is stored in that String may or may not make sense without extra processing. Having to account for these different scenarios adds complexity to systems that can cause implementation problems as networks grow. Therefore, having standard data types reduces the number of scenarios our systems need to support.

With Haystack 4 we undertook addressing the next level of sophistication in semantic modeling – developing a taxonomy and an ontology to support the ability to represent machine-readable relationships of things, their data and each other.

By Taxonomy we refer to a way of defining the relationships of things. For example, we say that water is a subtype of liquid because it is a specific type of liquid. The converse is that liquid is a super-type of water. Haystack 4 utilizes the concept of subtypes to organize all terms into a tree-based taxonomy. This provides us with defined and agreed upon relationships of things. It also guides users in applying tagging in a uniform way.

By Ontology we refer to the way a semantic model captures relationships between things, such as which Air Handling Unit feeds air to an occupant space. A structured taxonomy in the key step in achieving the benefits of a rich ontology of devices and equipment systems. A powerful use case is tracking the flow of energy across systems. The energy could be used to convey heated or cooled gas through a duct or liquid through a pipe, but without a standard way of representing the flow of energy, or any relationships between things, we can’t drive the industry forward by making our tools more capable of automatically analyzing these relationships. Haystack 4 extends the standard to support the implementation of both a taxonomy and the resulting ontologies.

Support for RDF/Linked Data – Another major new feature of Haystack 4 is support for RDF. RDF (Resource Description Framework) is a family of World Wide Web Consortium (W3C) specifications originally designed as a metadata data model. It has come to be used as a general method for conceptual description or modeling of information that is implemented in web resources, using a variety of syntax notations and data serialization formats. [Wikipedia]. It provides ability to accomplish semantic modelling but is typically the domain of software engineers and is not very accessible by industry professionals dealing with the real world of equipment systems.

When originally creating the Haystack approach, we felt it was important that modeling of equipment systems not require users to understand advanced software modeling concepts as a starting point. Rather, Haystack allowed users to focus on the more tangible “facts” or descriptors about data and equipment that they readily understood. Haystack 1 took a much simpler and more accessible approach to enable industry practitioners to describe the characteristics of the equipment systems and devices that they encountered by adding simple descriptive tags. This made Haystack very accessible to industry professionals and has been a key reason for its widespread adoption and success.

As the understanding semantic modeling of device and equipment data has matured, users are seeing interest in taking advantage the techniques and capabilities available with RDF. This allows software developers to utilize Haystack with the RDF techniques and semantic modelling tools they may be familiar with, without losing compatibility with the tag-oriented approach typically used by industry practitioners and tools designed for use by the people in the field.

Q: Can anyone do it? What are the requirements?

A: Yes, anyone can use Haystack. The only “requirement” is to understand the basic concepts of semantic modeling as we have outlined above. All of the work of the Haystack community is open source. There is no cost to access it and no ongoing licensing cost. You do not even have to register to download all of the work that the community produces! Anyone is free to participate in discussion forums and working groups as long as contributed IP is licensed under the Academic Free License 3.0. This ensures that Haystack IP is open and freely available for any commercial use.

Another key point is that the Haystack methodology can be applied without needing to know anything about software or programming. For example, you can apply Haystack tags to a handwritten list of data items with a pencil and paper, in spreadsheets where the tags are added as columns, in menu driven software applications where tags are defined in dialog boxes, to programmer-level implementation.

Q: What does it take to build systems and equipment based on project Haystack?

A: As far as implementing Haystack in products and software applications, the community has developed a wide range of reference implementations in a variety of common languages, all provided as open source. Those have proven to be very effective in assisting programmers to get up and running with Haystack quickly. The core concept that needs to be implemented is a way to add attributes (tags) to records in the software application. Most applications have this as a fundamental part of their design. The development effort is to transition that to follow the standardized Haystack model. Beyond the reference implementations, the Haystack developers website provides extensive documentation.

The community also maintains a market focused website with articles, the highly successful Connections Magazine, Guide Specifications and other supporting materials.

Recommended: StaffingScore Provides 360° Performance Evaluations For Temporary IT Staff

Q: What are your plans for the future?

A: With Haystack 4 the technology is all in place to apply standard Haystack semantic modeling to virtually any type of equipment, device or sensor. The major efforts are to educate the industry and foster the interaction and growth of the community to address more and more systems via Haystack Working Groups and share that knowledge among the community.

As far as the “future state”, by employing Haystack semantic tagging software applications can automatically find and interpret the data they need to automatically provide value to the user. As an example, software today can automatically generate equipment graphics and system views simply by interpreting tags on the control system data. Hours of manual graphics assembly is thereby eliminated reducing project cost and increasing value creation.

As another example, analytics applications can quickly consume data from equipment systems and interpret patterns in operational data to identify faults, deviations, and trends that can be addressed to improve efficiency and insure proper operation of equipment systems.

As we said at the start, Project Haystack enables a future where a push of a button can turn data into true intelligence reducing the cost of intelligent building systems and enabling operational teams to better understand and improve the operation and performance of buildings.