CAP and ACID Principles in Mobile Database Solutions

The concept of a mobile database brings some characteristics from distributed systems and incorporates the growing developments in wireless technology and mobile devices. At a practical level, a mobile database solution allows a business user to have connectivity to a corporate central database while in the field, without a dedicated link to the corporate database server. The method of communication is not unlike asynchronous replication, wherein the updates between mobile node and corporate node are handled on a schedule as opposed to instantaneously. According to Connolly & Begg (2015), mobile database solutions typically include:

  1. a corporate database server and DBMS that manages and stores the corporate data and provides corporate applications;
  2. a remote database and DBMS that manages and stores the mobile data and provides mobile applications;
  3. a mobile database platform that includes laptop, smartphone, or other Internet access devices;
  4. two-way communication links between the corporate and mobile DBMS.

Beyond the functions of a standard DBMS, mobile database solutions also require the ability to:

  1. communicate with the centralized database server through modes such as wireless or Internet access;
  2. replicate data on the centralized database server and mobile device;
  3. synchronize data on the centralized database server and mobile device;
  4. capture data from various sources such as the Internet;
  5. manage data on the mobile device;
  6. analyze data on a mobile device;
  7.  create customized mobile applications (Connolly & Begg, 2015).

Common issues in mobile database solutions include security, network partitioning, cellular handoff, and ACID transaction management (Connolly & Begg, 2015; Ibikunle & Adegbenjo, 2013). Security, partitioning, and handoff are all inherent in the mobile nature of the solution; that is, the idea of mobile nodes roaming around the country, with the connections being handed off between cellular towers as the user traverses a route, obviously carries with it the possibilities of signal loss or physical loss of an unsecured device.

ACID principles, which address the cluster as a whole entity, must be relaxed and adapted for mobile database solutions (Connolly & Begg, 2015). Assume for a moment that a large number of modifications are done by a user on the mobile node. Those transactions must be committed to the central database at next update, but if the connection is lost or weakened as a cellular signal is handed off, the batch may not complete. In that case, according to strict Atomicity rules, the entire set of transactions must be rolled back. This is not optimal and thus a new approach to Atomicity must be defined for mobile solutions. During that time, isolation is an issue, because a resource is blocked until the transaction is released. This also brings in the questions of consistency and durability: what happens when connectivity is lost and the mobile database is inconsistent with the central database? It is not discoverable until the mobile database is able to re-establish connectivity. What happens if one or more components of the mobile database solution experiences a failure?

Similarly, as CAP concerns are raised when a network partition occurs, we must take into account additional partition likelihoods of mobile handoff or signal loss. The mobile database solution must choose between consistency or availability in an event of partition. In choosing consistency, none of the nodes will be available until they are all back online. In choosing availability, the nodes will be available but not necessarily consistent until connectivity is re-established between all nodes.

The shortcomings in both ACID and CAP are mostly relegated to CAP, which applies to the database solution as a whole. The system overall must be available. However, consistency is possible in a slightly more relaxed way (just as ACID properties tend to be more relaxed for mobile database solutions). Ramakrishnan (2012) acknowledges that consistency can exist on a spectrum.

References

Connolly, T., & Begg, C. (2015).  Database Systems – Database Systems: A Practical Approach to Design, Implementation, and Management (6th ed.). London, UK: Pearson.

Frank, L., Pedersen, R. U., Frank, C. H., & Larsson, N. J. (2014). The cap theorem versus databases with relaxed acid properties. Paper presented at the Proceedings of the 8th International Conference on Ubiquitous Information Management and Communication, Siem Reap, Cambodia.

Greiner, R. (2014). Cap theorem: Revisited.  Retrieved from http://robertgreiner.com/2014/08/cap-theorem-revisited/

Ibikunle, F. A., & Adegbenjo, A. A. (2013). Management issues and challenges in mobile database system. International Journal of Engineering Sciences & Emerging Technologies, 5(1).

Shapiro, M., Preguiça, N., Baquero, C., & Zawirski, M. (2011). Conflict-free replicated data types. Paper presented at the Stabilization, Safety, and Security of Distributed Systems, Berlin, Heidelberg.

Ulnes, S. A. (2017). Eventually consistent: A mobile-first distributed system.  Retrieved from https://academy.realm.io/posts/eventually-consistent-making-a-mobile-first-distributed-system/

Breaking down Synchronous and Asynchronous Database Replication

Synchronous writes the data at the same time across source and target(s), simultaneously. It is a single transaction, and all-or-nothing. Asynchronous writes to source then propagates the changes to the target(s) at regular intervals. It is on a schedule, so there is a lag between local commit and replication to remote nodes. This is most commonly used in cloud backup situations.

Breaking it down to the host-storage relationship:

Synchronous

  1. Source Host sends write request to Source Storage
  2. Source Storage writes data and sends to Target Storage
  3. Target Storage writes data and sends acknowledgement to Source Storage
  4. Source Storage acknowledgement to Source Host

Asynchronous

  1. Source Host sends write request to Source Storage
  2. Source Storage writes data and sends acknowledgement to Source Host
  3. The update is held in queue until a specified time, at which the Source Storage sends the update to Target Storage
  4. Target Storage writes data and sends acknowledgement to Source Storage.

The key difference in the chain is where/when the acknowledgement is sent. In Synchronous, the write-to-target action must complete successfully before a Source Host receives acknowledgement. In Asynchronous, the acknowledgement is sent upon writing to source, without confirming an immediate write to target.

Primary concerns between these two methods are data integrity and performance. Synchronous may guarantee no data loss but it consumes much more bandwidth and cost than an Asynchronous solution. Asynchronous tends to be more cost-effective and uses less resources, and is more resilient by design; however, data loss at write is more likely. If data must match real-time across nodes, Synchronous Replication is preferable, as even a small delay between local and remote write in Asynchronous Replication may not be allowable.

References

Connolly, T., & Begg, C. (2015).  Database Systems – Database Systems: A Practical Approach to Design, Implementation, and Management (6th ed.). London, UK: Pearson.

Vembu. (2016). Synchronous (vs) Asynchronous Replication. Retrieved from https://www.vembu.com/blog/synchronous-vs-asynchronous-replication/

Competing Values: The “Why” Behind the “What and How”

In my current pre-research for eventual dissertation work, I explore the inability of companies to capitalize on analytics capabilities due to a lack of a data-centric culture, and seek to identify a number of key measures for implementing such. This is admittedly an interdisciplinary endeavor. Büschgens, Bausch, & Balkin (2013) focus on the broader organizational culture phenomenon, but their theoretical approach and meta-analysis are relevant. The same principles that foster a successful organizational culture may also be useful for implementing different subcultures or niche cultures that are a part of that broader successful culture.

The authors introduce two important theoretical constructs: Measurement of Behavior and Output (Ouchi, 1979) and the Competing Values Framework (Quinn and Rohrbaugh, 1983; Quinn and Spreitzer, 1991). Ouchi’s model, for our purposes, begins with a low ability to measure outputs. Thus, the organizational process is classified on the knowledge of the transformation process. For implementing a data-centric culture, our hope is that stakeholders are closely engaged, but that is not always within control. Even in a worst-case scenario (or, clan control), it is possible to “[align] the individual’s objectives with those of the organization” (Büschgens, Bausch, & Balkin, 2013, p. 766).

The Competing Values Framework is a useful tool for quantifying the specific means and ends each part of an organization most closely identifies with. This is of particular importance when a data-centric culture spans over multiple internal entities. Finance might be more Hierarchal in their approach, but IT may be more Rational. Appealing to why the data-centric culture is important will require different foci for each department based on their plot on the Competing Values Framework. Such is the focus of the meta-analysis. The authors investigate the relationship between innovation and the four major cultural traits, and outline their findings. Of particular interest are (a) those findings on the relationship and (b) the fact that “organizations that create radical innovations do not exhibit different organizational cultures than those that are rather oriented at incremental innovations” (Büschgens, Bausch, & Balkin, 2013, p. 775). This is encouraging, as organizations may sometimes feel overwhelmed or pushed by the need to make great strides in change when in fact the current climate would not support such radical change, and such a speed is accessible and relevant to all the major cultural types.

Those of us in IT would be wise to don a management consulting hat once in a while, seeking to understand our customers and what why drives their daily productivity.

References

Büschgens, T., Bausch, A., & Balkin, D. B. (2013). Organizational culture and innovation: A meta-analytic review. Journal of Product Innovation Management, 30(4), 763–781. doi: 10.1111/jpim.12021.

Ouchi, W. G. (1979). A conceptual framework for the design of organizational control mechanisms. Management Science, 25(9), 833–48.

Quinn, R. E., & Rohrbaugh, J. (1983). A spatial model of effectiveness criteria: Towards a competing values approach to organizational analysis. Management Science, 29(3), 363–77.

Quinn, R. E., & Spreitzer, G. M. (1991). The psychometrics of the competing values culture instrument and an analysis of the impact of organizational culture on quality of life. Research in Organizational Change and Development, 5, 15–42.

This post originally appeared on my LinkedIn page: https://www.linkedin.com/pulse/competing-values-why-behind-what-how-jonathan-fowler/

Data-Driven Cultures, Leadership, and Organizational Change

Feldman (2004) outlines ten principles of research that make meaningful contributions to the field and are grounded in theory in some way. Some of these principles are fairly obvious: the research question is not trivial, the author shows mastery over the research process, the writing is of a particular quality, and it goes beyond synthesizing research to provide new insights. Other principles go beyond what we might consider obvious needs. First, there must be a tight balance between including too much in a literature review and being sparse. That balance demonstrates an understanding of what is relevant to the problem. The writing must also demonstrate a purpose, introducing new perspectives that make a difference in the field. It has clear focus and a specific theoretical domain (or domains, if necessary). New constraints are identified and there is a clear relationship between independent and dependent variables. Feldman (2004) also outlines three key questions of a solid paper: (1) Is there anything surprising here? (2) What theories can be used to explain it? (3) Can the theoretical perspectives be integrated?

Much has been made over the impact of business intelligence and a data-driven culture in organizations. Where leadership is concerned, the discussion typically has revolved around how executives in the organization must buy in to a company-wide culture change and implement new frameworks to support analytics at all levels of the organization. This is primarily a top-down view of the relationship between business intelligence, data-driven culture, and organizational leadership. However, very little research exists on how data-driven culture affects organizational leadership—from the bottom up, in other words—beyond empowering better decision making with improved metrics. This is an area open to much more investigation.

That is not to say that the topic has not been approached. Marshall, Mueck, and Shockley (2015) argue that organizational leaders “embrace analytics and actionable insights” but also integrate “analytics and insights with innovation” (p. 33). This is clearly a call for a two-way relationship between leadership and the data-driven culture. Leaders must not only drive the organization’s embrace of data-driven culture but be willing to be affected by it.

More recently, McCarthy, Sammon, and Murphy (2017) directly question how data-driven cultures affect leadership styles. Although the article is focused on higher education institutions, it is applicable across companies. It provides equal discussion on both data-driven problem solving (in this case, student retention) and leadership studies. A thorough review of leadership styles is distilled into 3 broad groups: task-oriented, relations-oriented, and change-oriented. Task-oriented focuses on short-term planning and role clarification. Relationship-oriented focuses on developing, supporting, and empowering individuals within the organization. Change-oriented focuses on larger shifts in company thinking, taking risks, and external monitoring. The authors find significant evidence of data-driven cultures moving away from task- and relationship-oriented styles and towards change-oriented styles.

This introduces a symbiotic relationship between data-driven culture, business intelligence, and a specific change-oriented leadership style. Power (2016) discusses the relationship between business intelligence and decision making, reminding us that although data scientists “provide support for managerial decision-making” (p. 354), the business intelligence efforts are there for support, and organizational leadership must “continue to assume personal responsibility for their choices and organisational actions they initiate” (p. 350).

McCarthy et al. (2017) demonstrate the relationship between data-driven culture and change-oriented leadership. It stands to reason that an organization focused on fact-based decision-making (leveraging corporate resources in a more accountable way) would move towards a leadership style that favors change management over taskmaster management or relationship-driven management. That isn’t to say that these other styles are out of place; in fact, further research would be appropriate in different organization types and varied business domains. The discussion of particular case studies by Marshall et al. (2015) suggest an openness to other leadership types within data-driven culture.

It is apparent that insufficient research exists on the relationships between data-driven culture, analytics capabilities, and leadership styles. My research interests around data-driven culture and BI maturity comprise a sort of domino effect across multiple domains and issues. In this case, we may think of a data-driven culture running parallel to whatever leadership style is present in the organization; from there, a company’s information resource governance would be a product of the two.

References

Feldman, D. C. (2004). What are we talking about when we talk about theory? Journal of Management, 30(5), 565–567.

Marshall, A., Mueck, S., & Shockley, R. (2015). How leading organizations use big data and analytics to innovate. Strategy and Leadership, 43(5), 32-39.

McCarthy, J., Sammon, D., & Murphy, C. (2017). Leadership styles in a data driven culture. Paper presented at the European Conference on Management, Leadership & Governance, Kidmore End.

Power, D. J. (2016). Data science: Supporting decision-making. Journal of Decision Systems, 25(4), 345-356.