Data-in-Motion or Data-at-Rest?

Reading the available material on data-in-motion reminds me of when I first read about data lakes over data warehouses, or NoSQL over SQL: the urgency of the former, and outright danger of the latter, are both overblown. Put simply, data-in-motion provides real-time insights. Most of our analytics efforts across data science spheres apply to stored data, be it years, weeks, or hours old. Taking a look at data-in-motion means not storing it prior to analyzing it, and extracting insights as the data rolls in. This is one workstream of dual efforts that tell us the whole picture: historical data providing insight on what happened and potential to train for future detection, and real-time data to get what’s happening right now.

Churchward (2018) argues a fair point here: once data is stored, it isn’t real-time by definition. But taking that argument to a logical extreme by asking whether we would like to make decisions on data three months old is a stretch. While it is true that matters such as security and intrusion detection must have real-time detection, categorically dismissing data-at-rest analytics is reckless. It vilifies practices that are the foundation of any comprehensive analytics strategy. Both data-at-rest and data-in-motion are valuable drivers of any business intelligence effort that seeks to paint a total picture of any phenomena. 

There are, of course, less frantic cases to be made for data-in-motion. Patel (2018) illustrates a critical situation on an oil drilling rig, in which information even a few minutes old can be life-threatening. In this case, written for Spotfire X, there may be some confusion of monitoring versus analytics. The dashboard shown on the website and written scenario paint more a picture of monitoring and dashboarding than the sort of analytics we would consider deploying Spark or Kafka for. I don’t need a lot of processing power to tell me that a temperature sensor readings are increasing.

Performing real-time analytics on data-in-motion is an intensive task, requiring quite a bit of computing resources. Scalable solutions such as Spark or Kafka are available but may eventually hit a wall. Providers such as Logtrust (2017) differentiate themselves as a real-time analytics provider by pointing out the potential shortfalls of those solutions and offer a single platform for both data-in-motion and data-at-rest.

References

Churchward, G. (2018). Why “true” real-time data matters: The risks of processing data-at-rest rather than data-in-motion. Retrieved from https://insidebigdata.com/2018/03/22/true-real-time-data-matters-risks-processing-data-rest-rather-data-motion/

Logtrust. (2017). Real-time IoT big data-in-motion analytics.

Patel, M. (2018). A new era of analytics: Connect and visually analyze data in motion. Retrieved from https://www.tibco.com/blog/2018/12/17/a-new-era-of-analytics-connect-and-visually-analyze-data-in-motion/

Challenges of Health Informatics in the Cloud

Alghatani and Rezgui (2019) present a framework for remote patient monitoring via cloud architecture. The primary intention is to reduce disparate data sources and walls between various data siloes, increasing cost effectiveness, response time, and quality of care. The cloud architecture involves the database itself, user interface(s), and artificial intelligence. This cloud is used by four primary groups: patients, hospitals, insurance companies, and controllers (system stewards).

The authors outline a number of advantages here. Telemedicine can be a great thing but has a number of barriers to overcome, not the least of which are cost, culture, political environment, and infrastructure. The cloud architecture seeks to mitigate the cost and infrastructure issues. IT resources can be extended dynamically based on need and the decentralized nature of the system allows for better scalability, flexibility, and reliability.

There are a number of challenges to be considered. The authors highlight seven:

  1. Security
  2. Data management
  3. Governance
  4. Control
  5. Reliability
  6. Availability
  7. Business continuity

An extensive discussion on data collection challenges is presented, outlining a number of possible methods for collection and synchronization. There must be an assumption that no device on this architecture will maintain constant contact with the cloud, and consistency models must be taken into consideration. One option is for each device to maintain local storage and upload to the cloud once a stable connection is available. Another option is a whisper network of its own, much like the early Amazon Kindle devices. A third and final option—also the authors’ proposal—is the utilization of fog computing as a layer between these devices and the cloud.

Privacy is always an issue and cloud architecture muddies the waters a bit, as there is no on-premise server locked down that holds the personally identifiable information. Banks and hospitals have typically been the slowest to adopt cloud computing, in my experience. As Alghatani and Rezgui (2019) note, governance and control are concerns here. The Health Insurance Portability and Accountability Act (HIPAA) requires confidentiality in all individually-identifiable health information; in 2013, this law was extended to genetic information by way of the Genetic Information Nondiscrimination Act (GINA). While the rules prohibit use of genetic information for underwriting purposes, there is no restriction on the sharing or use of genetic information that has been de-identified (National Human Genome Research Institute, 2015). De-identification is not entirely foolproof. There are cases in which the data can be re-identified (Rosenbaum, 2018).

References

Alghatani, K., & Abdelmounaam, R. (2019). A cloud-based intelligent remote patient monitoring architecture. Paper presented at the International Conference on Health Informatics & Medical Systems, HIMS’19, Las Vegas, NV.

National Human Genome Research Institute. (2015). Privacy in genomics. Retrieved from https://www.genome.gov/about-genomics/policy-issues/Privacy

Rosenbaum, E. (2018). Five biggest risks of sharing your DNA with consumer genetic-testing companies. Retrieved from https://www.cnbc.com/2018/06/16/5-biggest-risks-of-sharing-dna-with-consumer-genetic-testing-companies.html