At work, do you prefer Email, IM or just Bonnet-Meetings?

A discussion of generational communication challenges

John is an accomplished experienced lawyer. On a recent trip together, he lamented being increasingly exasperated by some of the young 'uns in his office.

"I sent him a detailed email with the entire legal position last week," John told me, "But on Monday the guy tells me that he never saw anything. He just said he doesn’t read emails!"

"What am I supposed to do? How should I be informing him of what he needs to know?" John asked me.

John's right. If I send you a communication and you don’t even want to open the tool, what can I do? Is reading emails mandatory? Is that a formal policy? In what time period should emails be read or responded to? Surprisingly, not many organizations have any such policy or clarity on this issue.

We are at a curious cusp. The different ways in which we can communicate with each other has grown exponentially. And we have established no rules for any of them! When should we use what? And whose job is it to create this guidance anyway? HR? IT? Management? It is not entirely clear.

I know that there are those who will see this attempt at creating clarity as more bureaucracy and an unnecessary attempt to formalize something that works just fine informally. Perhaps it does. But as a technologist interested in the most efficient use of these tools, I see more communication breakdowns with the increased number of tools - not necessarily better communication.

In most enterprises today, we have some if not all of the following means of communication: phones, meetings, emails, instant messages, social media, snail mail, faxes, memos etc. Admittedly, some of those at the end of that list are less frequently used, and I probably missed a few others. Carrier pigeons?

Different generations – GI generation (Born 1901-1926), Mature Silents (Born 1927-1945), Baby Boomers (Born 1946-1964), Generation Xers (Born 1965-1980), Millennials or Generation Y (Born 1981-2000, and Boomlets or Generation Z (Born after 2001)- don’t use these communication tools in the same way. This is significant in environments where many generations are intermingled.

Here is one blueprint for usage. Admittedly, this is a work in progress and I welcome your comments and suggestions

EMAIL

o   Do's

§  Use email only when you have no expectation of an immediate response. This is important.

§  Some experts now recommend checking emails infrequently, say once an hour, or even just a few times a day. Otherwise, as many studies have shown, we have far too many disruptions to perform focused work.

§  If you are trying to connect to a GI or Baby Boomer, use email. You have a better chance of being noticed.

o   Don'ts

§  If your email is longer than 4 reasonable sized paragraphs, you need a phone call or a meeting (that number is fungible, but not by much)

§  If you are one of those whose email end up with six colors, as it moves back and forth with a new color for every response - you don’t understand email! Multi-colors are for whiteboards not emails.

o   And finally, many experts are beginning to look at email as a legacy application. Remember memos....

INSTANT MESSAGE or IM aka “Texts”

o   Do's

§  Use IM when you have a "quick" question or want to respond with a "short" answer.

§  If you are trying to connect to a Gen Xer or Millennial, you are far more likely to get a response on an IM.

o   Don’ts

§  If your IM is more than a sentence, you are probably not using it correctly.

§  It is not a substitute for an email.

§  Just because you see someone's presence showing green, does not mean they are doing nothing and will respond in 5 seconds (presence allows you to see availability)

§  Don’t assume a Baby Boomer is paying attention to your "texts" (many will call them that)! They probably don’t.

  MEETING

Contrary to common perception, meetings are evolving.

o   First, the 'traditional' meeting, the best option for discussions and information sharing, especially when provisioned with advance agendas and post meeting deliverables. Most popular with the older generations.

o   Another obvious variation is the sidebar or impromptu, where you walk to someone's office or cubicle to discuss something that is important and urgent. There is also the rise of the "teaming room" meetings.

o   Video/audio conferences are popular in an environment of virtual and telecommuting employees.

o   Next, we have the 'stand-up' or 'bonnet' meetings made popular by tech companies. These are quick check-ins, and participants will stand in a circle or around a table ('bonnet' is another word for the hood of a car). More popular with the younger hip crowd.

o   And finally, the 'disrupters' have created yet another form of meeting made famous at many large and small companies - the 'optional' meetings. Yes, in certain companies every meeting is optional!  Bain & Company found that the average manager wastes nearly a full day every week in meetings, and senior executives waste a staggering 40% of their work week in endless, mind-numbing, and ineffective meetings. Not surprisingly, these optional meetings are quite successful. Radical!

PHONE

A phone call fits somewhere between a meeting and an email. There are some conflicting signals about the use of voicemail versus email. In many organizations, leaving a voicemail does not necessarily elevate the communication to an urgent status. Many use voicemail as the "spam call" repository.

o   Do's

§  The older generations tend to use phones more often and respond better to calls.

§  This communication is generally more urgent.

o   Don'ts

§  Just because you know a coworker's mobile phone number does not mean you should use it. Being mindful about using phones for work particularly after-hours, weekends etc. is a good idea.

§  Younger generations are far less enamored by talking to you.

MEMO - ANNOUNCEMENTS - BLOGS

This is one area that has undergone quite a bit of change.

o   Do's

§  Communications on organizational policies etc. are now located more frequently in 'announcements' or posts on internal enterprise social media (ESM) outlets like Yammer etc.

§  The younger generations find this more convenient but the older generations are still stymied by social media.

o   Don'ts

§  Memos are passé as the repositories of "lengthy communications" – and are increasingly shunned as a means of effective communication.

§  Many of the older generations find this void troubling and invariably try to use emails for that purpose, with mixed results.

§  Frankly, the attention spans of the new entrants into our work environments and their preference for shorter communications - has been a death-knell for memos. Viva la Twitter!

ENTERPRISE SOCIAL MEDIA (YAMMER, CHATTER, HANGOUTS ETC.)

This is the newer entrant and again the harder shift for older generations.

o   The problem is best understood as a "push" versus "pull" conundrum. Memos are pushed out. Social media is generally understood as a pull. A la carte is out. We are all now at a buffet. Learn to find it, otherwise you may go hungry.

o   The younger generations, familiar with the Facebook and Twitter worlds, have had no problem with this concept and use. The older generation, responsible for much of the content, is instead trying to use social media as a substitute for email.

Given the tremendous challenges of this plethora of choices, vendors are experimenting with all manner of hybrids.  Will more options makes this easier?

On July 22, 2015 Microsoft announced Send. Send is a cross between an SMS and an email. As Microsoft describes it: It is a brief, snappy communication. It is when you want an instant response, and yet want to be able to retain a record of what you said in Outlook. Microsoft says Send is different from regular email because it only shows messages sent using the app itself. Time will tell if this will work.

Many organizations spend a lot of money and time on personality testing and team building. It is not uncommon to walk into enterprises and find the Myers-Briggs profile alphabets of the staff prominently placed in their workspaces. It helps to understand a colleague’s personality. Is it time to add a "communication preference" to that alphabet soup?

"I prefer ESM, IM & Bonnets". You?

When did my car become a hackable computer?

With the host of recent news media accounts of sophisticated hacks on all manner of cars, there is a sense of shock and surprise.

“Are you suggesting that my Jeep can start, stop and turn without me doing anything? Wow! When did this happen?”

Well, actually this shift has been in motion for a while. We just haven’t paid any attention to it and it exposes a reality that is worth exploring.

However, to understand this,  first we have to understand that computers operate on code. Code is written by human beings in lines, just as this blog is written in lines. The computer then consumes the Lines of Code that then tell it what actions to perform. 

Lines of Code

Now, let’s look at a timeline:

• Apollo 11 in 1969 had approximately 145,000 Lines of Code in the so-called Apollo Guidance Computer (AGC). That computer enabled astronauts to go to the moon and come back safely. Even so, Neil Armstrong and Buzz Aldrin overrode their Lunar Module computer when it almost landed them in a crater! But in terms of power, it was more basic than the electronics in modern toasters that have computer controlled stop/start/defrost buttons. The AGC had approximately 64Kbyte of memory and operated at a paltry 0.043MHz.

Apollo 11 computer

• The space shuttle's General Purpose Computer (GPC) was originally designed in January 1972. NASA chose state-of-the-art flight computers from that era, and it had approx. 400,000 Lines of Code. It was upgraded many times and the Lines of Code went up with every upgrade.
• Microsoft Office has approx. 45,000,000 Lines of Code! A quantum leap in complexity. Microsoft regularly patches and attempts to fix this code. Not always successfully.
• CERN’s Large Hadron Collider that recently found the elusive Higgs Boson had approx. 50,000,000 Lines of Code before the most recent upgrade to find even smaller quantum particles.

Large Hadron Collider

• A modern automobile, by comparison, has approx. 100,000,000 Lines of Code! Whoa, wait a minute –my car has twice as many Lines of Code as the Hadron Collider? Yes. Depending on your car, it could have even more than that.

Why are these Lines of Code important?

According to a Carnegie Mellon report, there are 20-30 bugs per 1 to 1.5 million Lines of Code. Do the math. When you have millions of Lines of Code, there are more mistakes and hence more bugs. Bugs make the code more vulnerable – more hackable!

The idea that software – even from the biggest companies like Chrysler, Toyota, Google, Microsoft etc. – comes with NO bugs, is an illusion. Some companies have understood that reality and are attempting to get ahead of this threat. These companies are no longer in denial. They include software giant Google that routinely pays hackers millions of dollars for telling them of bugs in their software. More companies are getting in on this crowd-sourcing model for exposing vulnerabilities in their software. Tesla, the electric car maker, is already upgrading and updating their cars wirelessly over the internet. But for all those that are attempting to correct the code, many more have very poor security practices in the writing of that code and shabby quality control. An entire cadre of attorneys is salivating at the prospect of suing all these companies for the pain they will be causing in the future.

United Airlines recently offered and then paid two hackers one million frequent flier miles each for spotting issues with their software. One million miles is the equivalent of 33 round trips from USA to Europe!

Daimler-Chrysler, BMW and Audi are buying Nokia maps for $2.7 billion. Dieter Zetsche, chief executive of Daimler, said a desire to have better control over data security was one of the reasons Mercedes was bidding for Nokia's high-definition mapping business. Dieter wants the Mercedes to steer where the driver intends the car to go and in the new and fast approaching era of driverless cars – this will be even more important. By the way, the Lines of Code in driverless cars is expected to grow exponentially, as then will the bugs.

Computers run Lines of Code. Code is not perfect. And whereas imperfect code in your laptop may not kill you (although that may not be as simplistic as that statement suggests), the computer you drive and call a car, certainly can.

Why no one is talking about the scariest part of the Sony hack?

At some point a major Hollywood studio decides to make a silly movie depicting an assassination attempt on the dictator of the last reclusive country on earth - North Korea. (Please remember the last two words of the last sentence. They could be a clue later on).

On November 23, the studio that made the movie is hacked. For two days their employees work with no computers. On November 27, four unreleased movies from the same studio are dumped onto the internet and millions watch them.

On November 28, reports suspect North Korea because a North Korean website calls The Interview “an evil act of provocation.” Really? Could that be possible? What would provoke them? What a crazy thought?

On December 2, nine days after the breach, Sony chiefs Michael Lynton and Amy Pascal issue an alert to employees about the attack: “It is now apparent that a large amount of confidential Sony Pictures Entertainment data has been stolen by the cyber attackers". What? It took six days to figure out that the unreleased movies and others salacious details dumped onto the internet are yours?

And all this while the mighty agencies of the US government are also investigating -with no success. No one has a clue who did it. Really? Who were they trying to assassinate in the movie again? Oh, I get it. No one knows because that movie has not been released in theaters yet.

On December 7, North Korea denies involvement in the hack while praising it as a “righteous deed.” Indeed! That must have thrown all the government agencies off the scent of the real perpetrators. Exquisite diversion.

Now emails, social security numbers, medical records, more threats, passwords and much more come cascading down onto the internet. But who did this? What a mystery!

To add to this covert operation, on December 14, the script for the still to be filmed new James Bond movie is divulged by these mysterious hackers. At this point my 10 year old is asking me, "You have got to be kidding me! Mysterious hackers?"

On December 15, Sony's chief informs a stunned town hall that this is an investigation now being handled by "the highest levels of the FBI". It has now been 17 days and we still have no clue who did this.

On December 18, almost 3 weeks after the hack, the US government accuses North Korea of this cyber-attack on Sony pictures!

Why did a 10 year old figure out who did this before the FBI? That is the scariest part of this story.

Why does the man who led IBM’s Watson Project advocate technology timeouts?

Manoj Saxena is a man heavily immersed in technology. For many years he ran IBM’s famed Watson project. Today he is a successful venture capitalist and seeds many more technology companies encompassing the next wave of technology and innovation.

So, why then is he advocating that in the tech companies he now runs, staff have mandatory unplug days? A technologist preaching technology time outs? Blasphemous!

On the face of it, this may appear strange. But it is really not. In fact, it appears surprisingly well thought out. On predetermined holidays/dates his companies go into electronic silence. No emails, no texts, not even phone calls for work. Just time to relax, rejuvenate and reconnect with family and friends. Manoj postulates we – all of us – are rapidly approaching an electronic saturation point. Like others, he believes that the always-on and connected mantra is no longer the panacea it may have appeared to be. It is taking a toll. He wants to go even further and extend the technology moratorium to at least one of the weekend days, if not both.

This sentiment is shared with many other technologists including none other than Steve Jobs. Steve was apparently a self-described “low-tech” parent who restricted his own kids’ access to technology for some of the same reasons. In fact Jobs had such strong reservations about allowing his kids unfettered and perpetual access to technology that he confessed that his wife and kids accused him of being a fascist! The man who may have single-handedly brought touchscreens into our lives, had serious concerns about the long-term effects of engaging in touchscreen technology for extended periods of time. According to Walter Isaacson who spent many hours in the Job’s household while writing his book, face-to-face family time came before any screen time. No iPads were permitted at the dining table.

The always connected life-style takes many tolls – from cost to productivity to creativity. Every economics student knows that switching involves cost. Today, when multi-tasking with technology has become a virtual epidemic, why would we think it has no adverse effect?

Erica Fox, the best-selling author and Harvard professor recently wrote an excellent piece titled “Is Never Offline, good for business and life?” where she was reacting to the recent Time Magazine cover “Never offline”. She rightly points out that those who are never offline spend too much time reacting and not enough solving problems. They make decisions in a “frenzied and buzzing state of mind”. They are also becoming less and less familiar with focused attentive work and face-to-face relationships.

In a landmark report, Gloria Mack of the University of California at Irvine, found that typically a person in an office experiences a paltry 11 minutes before an interruption. And it then takes an average of 25 minutes before that person can return to the original task! What is the quality of work with such rapid and incessant switching generally driven by technology?

New research from the Carnegie Mellon University’s Human-Computer Interaction Lab shows that if you try to do two things at the same time, both things suffer! They found that the distraction of an interruption turned test takers in their lab to become on average of 20 percent dumber. It was enough to turn a B-minus student (80 percent) into a failure (62 percent).

According to Stanford sociologist, Clifford Nass,who conducted some of the first tests into the effects of multi-tasking, those who cannot resist doing two things at the same time are “suckers for irrelevancy”.  Apparently, we are not just suckers for that new text message or email, but it is actually making us stupid.

All this should concern organizations of every type. According to the WSJ, distractions cost companies money. One survey in 2011 found that businesses might be losing as much as $ 10,000 per employee every year due to distractions and poorly designed technology. Many among us answer every instant message, email or text at the very moment it arrives. And yet, how many companies have programs where employees are being counselled on the impact of multi-tasking and its effect on the company’s bottom line? Or how to manage the multiple avenues for interruption we all face in any office today?

In a recent book just released this September, “The Organized Mind”, Daniel Levitin (a professor of psychology at McGill University) explores many facets of our lives under the stress of information overload and multi-tasking. According to Prof. Levitin, multi-tasking puts us into a dopamine-addiction loop which is similar to cocaine addiction. Each time we do a new small task, our brains reward us with a tiny shot of dopamine, which is the chemical in our brains responsible for pleasure. He quotes a famous study in the 1950s where rats were given the opportunity to press a bar to get a shot of dopamine. Soon they were pressing the bar to the exclusion of everything including eating, drinking, sleeping and even sex! They died of starvation and dehydration. Levitin strongly suggests that we spend time away from our devices otherwise it will become like any other addiction.

So should we all abandon our devices? Not exactly.

According to Marshall Van Alstyne, an associate professor at Boston University and MIT, be very cautious about technologies like social media and those which interrupt you constantly, because that interruption dramatically reduces productivity. The solution, according to him, is to batch time and tasks. So, rather than checking your email every time the notification appears – turn the notification off, and check email once every hour or couple of hours. Don’t interrupt your research paper or memo every time the phone dings and informs you that someone has sent you a text. Turn the ding off. 

Manoj, Steve Jobs (and many other technology luminaries) are right. They were involved with the creation of the technologies which may have resulted in our distracted way of life. Many now recommend we need to adjust, and are calling for technology time-outs and less multi-tasking. They should know. And unlike many of us, some of them are actually doing something about it.

Why Project Managers need to understand EBay's Rating Systems

Something new and exciting appears to be happening in the world of projects. It may just be the beginning of an entirely new kind of project bicycle; one that comes with many more wheels than we have been accustomed to see.

The perfect storm of über-specialization added to the potent mix of ubiquitous internet access - is creating a new era of Micro Projects.

How are these different from the projects as we have known them?

Micro Projects are to projects what the factory assembly line was to manufacturing. Each component or task of the project is broken down to a far greater extent – many more wheels; the tasks are individually outsourced in an open marketplace of freelancers; these tasks may be completed by a person who specializes in only that one task; who may be located in a faraway place and speaks in languages other than English, and is someone you are unlikely to ever meet.

In this paradigm, every component, becomes a project with its own scope, vetting of the vendor, contract, acquisition, management, etc. It becomes a complete project - a Micro Project.

Micro Projects may have some unique characteristics which we have not encountered - at least not in the traditional way we have thought of projects:

1.    First, the units of work are very small- are hence far greater in number- and each unit can be completed by a freelancer.

2.    Such tasks are bid out to attract qualified independent contractors or freelancers who are increasingly flocking to this new landscape.  

3.    The buyers and providers of such Micro Project services are meeting in internet portals and sites setup to facilitate precisely this need.

4.    On some sites, freelancers are evaluated on the basis of work they have already done. They have ratings like a contractor might have on Angie’s List or perhaps a more sophisticated system like the buyer-seller ratings on EBay.

5.    As a buyer you may have a rating of your own for the contractors to evaluate as well.

6.    Payment is often handled via an escrow system like that on EBay where funds from buyers are held and dispersed to providers only upon task completion.

7.    Some even provide an arbitration process to assist with disputes.

This development is a natural evolution for certain kinds of projects where redundancy, costs, efficiency, timelines etc. are all positively affected as projects are broken down into far smaller and independent segments.

Imagine, for example, if the make-or-break tasks in the project (the Critical Path) are farmed out to multiple freelancers for the same task? Would that ensure a higher percentage completion, or quality, or timeliness? Call it “Micro Project Insurance.” If a particular outsourced task is extremely critical, why not have a backup plan and get two vendors to complete it independently? The Micro Project’s size of components reduces the individual component cost, thus making redundancy more cost effective.

In this model, the role of Project Managers becomes more of a broker involved with sourcing, managing, and integrating a rich dispersed vendor channel who are completing ever smaller elements that together embody the totality of the project. The emphasis shifts to detailed scope and task requirements, the vetting of freelancers, and multitudes of contracts to manage and administer.

The marketplace of websites facilitating this type of project, is proliferating. And more of these sites are appearing every day. There is a strong possibility that this may lead to a specialization of such sites. You may go to one site for marketing and another for designing a web site, etc. (see a short list of sites at the end of this blog).

Micro Projects are currently penetrating and altering traditional projects dealing with IT services, software development, design, marketing, communication and even administration projects. Over time, there is no reason to believe that other industries will not embrace this new breed of project.

As Micro Projects become more commonplace, Project Managers best be paying attention to riding and steering this new kind of bicycle. With many more wheels spinning under you, maintaining a common direction and making sure you stay in your seat, will take some new skills and much more dexterity.

The list below is but a small subset of the rapidly growing number of sites catering to this new phenomenon.

https://www.mturk.com/mturk/welcome Yes, Amazon is already in this game. Their tag line is: “We give businesses and developers access to an on-demand, scalable workforce”. Mechanical Turk is for HIT or human intelligence tasks and over 558,000 such tasks are currently available for freelancers.

http://www.Guru.com is another site where you can find freelancers to do tasks in writing, translation, web, design, multi-media, software, IT and admin tasks.

https://www.mycrowd.com MyCrowd, a startup that helps companies find freelance talent for specific tasks through Google Docs, PowerPoint and other software applications, is raising $500,000 in a seed round from unnamed angel investors to add more connectors and expand beyond its existing base of around 10 million specialists.

http://www.freelancer.com is a site which matches requirements in mobile application development, design, analytics, web design etc.

http://www.projects4hire.com is a plain vanilla site but has a very long list of engineering, IT and other service categories.

http://www.codingcupboard.com/ matches businesses with the best student coders.

https://pickcrew.com/ has a different angle. They fully vet every designer and developer and focus on web and mobile projects.

http://www.iwebmanage.com/content/bidforprojects/ has a built in escrow function and enable freelancers to even video conference on projects with employers. They will even assist project sponsors with a business analysis service to make it easier for them.

http://www.joomlancers.com/ is another site allowing buyers and providers to find themselves.

https://www.elance.com is one of the oldest sites for this purpose and had been around since 1999. They are more expensive due to their 10% fee and thus attract higher budget projects.

http://www.odesk.com  has an online tracking system, which makes it stand out from the crowd. This system checks employees' work every ten minute so that their work is assured to be under progress.

http://www.scriptlance.com  has no fees: free registration and free monthly services. In general, this is a competitive marketplace with hundreds of projects posted every day and thousands of freelancers bidding.

http://www.ifreelancer.com  registered freelancers have to pay membership fees for the services. 

http://www.fiverr.com/ here services start at $5.

http://www.outsourcing-partners.com has published rates for some tasks and focuses on software and design development as well as marketing projects.

http://worknhire.com is also focused on IT services.

https://www.origondo.com/ matches projects in marketing, design, communication, administration etc.

http://www.wordpressfreelance.com/ is a marketplace for those wishing to create presence in the WordPress environment. 

http://www.bizreef.com launched in October 2007 and hence once of the oldest, has over 25,000 freelancers vying for your business. Bizreef also has an Escrow system that guarantees that freelancers will receive order services as well as protection of their rights

Why Einstein might call Big Data “spooky”?

Human beings, and scientists in particular, are conditioned to look for causes. One of the most brilliant scientists of our time, Einstein, had his nemesis in what we call quantum mechanics (or quantum physics as it is also sometimes called). This is the study of the smallest particles known to man. Einstein’s theory of relativity and quantum theory are – in many ways – contradictions, which have still not been satisfactorily reconciled into one cohesive theory.

For example, at the quantum level, one particle can potentially be at two places at the same time! It defies all logic and yet it is possible. If you look at a nanoparticle, it can change from a wave form to a point. This is observable in the lab. Other types of strange behavior of these smallest nanoparticles – which in turn make up all things big and small - has been known for a long time and can be replicated. There is little doubt that this strangeness actually occurs.

Einstein himself called quantum mechanics “spooky” because we could all see what was happening, but were unable to explain how it is happening. Today, these issues are still unresolved.

How does this relate to Big Data? To understand this, let’s consider:

Until recently we lived in the era of “Small Data” which was characterized by the premise of exactitude. Databases were built to very specific formats and rules, so that they could be relied upon to retrieve accurate results which perfectly matched the queries we applied against such data. If you used a spreadsheet to compute data, this is what you did. Data was smaller in size – because among other things, computing and storage costs were very high. This in turn meant that databases had to be precise in order to be useful. Databases tended to answer the causal question: What is happening and why? It took time to use data this way and actionable intelligence was, in some cases, delayed. But, that was the cost of using small data and applying data analytics to it.

Today, the costs of data storage have shrunk dramatically. Computing power is enormous and climbing. Enter the era of “Big Data”.

One of the earlier examples of Big Data was in healthcare (as Viktor Mayer-Schönberger and Kenneth Cukier point out in their bestseller “Big Data”). Back in 2007, the CDC would go through precise collection of data points, aggregate them and then use that data to keep track of the spread of flu among the states. They had been doing the same thing for a while. All of this took time, data had to be collected from doctors’ offices and hospitals and by the time the CDC statistics were compiled and published, weeks went by and the data was sometimes already too old.

Around that time, Google was testing its algorithms and mathematical models to test search terms. They found that when users search for 45 terms in specific combinations – that became a predictor of the flu and was closely mapping the CDC data. Just like the CDC, Google too could now predict how flu was spreading nationwide, but unlike the CDC, their data was real-time! And Google did not have to go to doctors’ offices to get data, they extrapolated it from their users’ searches.

Google’s data model was not built on the concept of exactitude. Rather it was based on the use of patterns and correlations to tell us what was happening, not necessarily – how it was happening.

The Bureau of Labor is responsible for computing CPI or the Consumer Price Index. To do this, they employ thousands of people who report approximately 80,000 prices on everything from home rentals to the cost of airline tickets and everything in between. Many things are dependent on CPI including wages, social security amounts etc. CPI is very important. Again, by the time these numbers come out, they too are already old.

Then, two economists from MIT came up with a Big Data solution for CPI. They collected data on 500,000 products in the US – many more than the Bureau of Labor was collecting - and did so only using the web. Admittedly, their data was “messy” in that it was not exact – but it was possible to figure out a CPI equivalent much more quickly. And because they had much more data, any individual anomalies were less meaningful. This project is now a commercial venture and everyday thousands of financial institutions are making decisions based on this work.

In that sense, Big Data is the equivalent of quantum mechanics in this storyline. We can see what is happening, but don’t really know (or maybe even care) how it is exactly happening? The price of getting actionable intelligence in a timely way, is an absence of the precise causal questions and answers. Many are still uncomfortable with this lack of preciseness, and this is only the 2^nd or 3^rd inning of the Big Data story.  More discomfort is in store.

Today, a significant percentage of the trades on Wall Street are driven by computerized Big Data orders. Healthcare, retail, airlines, hotels – almost everything you can imagine – is being impacted by Big Data. We haven’t seen nothing yet.

In the era of Small Data, accuracy was preeminent because only small data sets were possible and had to be exact. But, as we delve deeper and deeper into this new world of Big Data, we are all making a tradeoff: the quality of the information becomes less important because the volume of the data evens things out. It is the speed of decision-making and the new insights we are learning which are becoming paramount, and all this is being revolutionized by Big Data driven by patterns and correlations.

Prof. Einstein would likely think Big Data too was “spooky” - because just like quantum mechanics - we know what it does, but don’t go looking for too many answers on exactly how and why it got there. You may be disappointed.

The problem with a Cyber-Posse

By some estimates, some data on one out of every two Americans has now been breached. So what can do you about it? Can you - as an individual or even as a corporation - go after those that may have compromised your data? If you are hacked, should you not be able to hack back against those who have stolen things from you?

Not so fast.

One of the more interesting discussions on the first day of the RSA Security Conference was: Does the law allow you to be the Cyber-Posse going after those that have compromised your systems and stolen your data?

The answer might actually surprise you. As a matter of fact, the law does not allow you to become a cyber vigilante. In the USA, you are expressly forbidden from doing just that. Section 18 USC 1030 (a) specifically prohibits this kind of activity.

When it comes to cyber exploitation, there is no room for affirmative defense in the law.

If after determining who has stolen your data, you decide to go after those criminals - perhaps simply to recover or delete your own data from their systems - you may still be on shaky legal grounds.If you were in Europe, you would hardly be better of. They too limit this kind of activity.

So there is a conundrum.

If someone stole property from your home you can go to the police. You may even have some rights to give chase if you apprehended someone in the act. If, on the other hand, someone stole your companies data, you don't have many choices. You cannot really go to the police (most local police are not equipped to deal with such a problem anyway) and in most instances you would realize the theft after the fact. And the law forbids you from hacking the hacker.

There is a convoluted logic to this law. What if you retaliated against someone who you thought had been the perpetrator? There is strong evidence that the hacking community takes great care to make sure that the overt signs on being discovered, are that someone else did it. If you accessed the systems of the wrong organization/person, thinking they were the ones who attacked and stole from you - you are now the perpetrator. Quite a mess and not one you can easily get out of.

And because you cannot chase down the hackers easily, and do so within the law, the hacking community is aware of that and may be able to use the existing laws to shield themselves from those who they have compromised and who might be wanting some retribution.

Don't hold your breath. No one is willing to take this on any time soon.

The only silver lining is that most prosecutors are turning a blind eye to someone who rides out on a Cyber-Posse. Prosecutors have other things to do and they are not too keen to be part of a headline which might read:

Prosecutor nails lady who tried to delete her own stolen data from the hacker's systems!