Predictive policing: the brave new age of law enforcement analytics

In the mid-1980’s I worked as an information analyst / data manager in a police station in a small town in New England (northeast U.S.).  It was a strange but exciting experience for a young man, chosen simply as I was identified as the local ‘computer geek’ at the town high school (a small and quite unenviable club – computer nerds were decidedly not ‘hip’ circa 1985).

Predictive policing

At the cusp of the age of personal computing and before the internet became widely available (access was via dial-up), the tools were primitive and simplistic. Still, by ritualizing the collection, storage, and retrieval of data, our little station began to change its approach to policing.

Working with the Police Chief, a wonderfully avuncular lawman, by turns jovial and admonishing, and a group of hardened, street wise detectives (one was on an undercover case and looked decidedly shifty in a Serpico way, with stubble, a tan, wide-lapelled patent leather jacket, and aviator sunglasses), we began consulting the statistics that issued from our seemingly magical IBM PC running DOS 3.1 and a rudimentary database controlled with painful and arcane command line instructions.

Old skool hi-tech: IBM PC running MS DOS circa 1985

For me, it was my early indoctrination not only to data analytics, but to the ability of data-focused computing to improve organizational decision making (thus, to the broader contest of business analytics).

Times have both changed radically, yet in some ways, very little. We take for granted the omnipresence of massively, inconceivably powerful (from the 1980’s context) computing tools, along with the amazing presence of the world’s combined knowledge via the prolific internet. Law enforcement now has tools so powerful for monitoring a populace that issues of power and privacy arise regularly. Beyond the debates, law enforcement officials struggle to understand how to deploy and integrate the wide variety of tools into their organizations.

As the power of analytics grows with increasing computing power and larger amounts and varieties of data, techniques previously relegated to academic social science researchers are now available to law enforcement professionals in the form of powerful software suites. From an analytics perspective, this means moving from static and structured ‘reports’ – traditional descriptive Business Intelligence – to predictive, prescriptive, and semantic analytics methods.

Analytics types (© Scott Mongeau 2014)

This also means moving from static models of social behavior to speculative hypotheses concerning complex social dynamics. Whereas increases in gun violence incidents can be plotted on a static timeline, predicting areas where gun violence is set to increase in order to improve preventative policing involves applying sophisticated statistical, machine learning, and optimization algorithms. Experts who combine the broad requisite skillsets – IT, programming, data management, statistics, social science / criminology, AND law enforcement fundamentals – are a rare breed.

Data science skillset. CC: Calvin.Andrus (2012)

The trend is to increasingly deploy sophisticated analytics solutions to amplify and improve law enforcement and intelligence services efficacy. The danger is to treat this as a problem of ‘shoehorning’ software into an organization. My advocacy, thinking back to those heady days many years ago working with my local Police Chief, is that the problem is primarily and fundamentally human and organizational. The challenge is to understand the goals and desired outcomes of law enforcement and intelligence agencies, per their legal and political charter, and to work with those professionals to design PROCESSES that assist PEOPLE to work effectively with the SYSTEMS. Thus, a human-centric and interface-focused approach to analytics solution design.

Integrated analytics solution design (© Scott Mongeau 2014)

With this in mind, what was applicable back then in ‘data analytics’ for policing that is still relevant today?

• Collecting data and ritualizing analysis builds discipline in decision making: The decision to comprehensively categorize, track, record, and examine crime incidents leads to focus and rigor in organizational processes.  By initiating the data analysis ‘program’ at our station, the Police Chief was signalling that he wanted to supplement ‘street smarts’ and intuition with careful consideration.  Although knowledge of cognitive decision biases was not as rich then as it is now, our Police Chief had a sense that we needed better foundations on which to build the case for improved policies and processes.
• Even simple descriptive statistical observations can lead to actionable efficiencies: Even simple statistical measures and time series analysis can be revelatory.  For instance, the observation of increases in drunk and disorderly incidents in a certain neighbourhood can serve to reinforce observations from beat cops, and so focus resources.  Otherwise, one is left with the risk of biases and distortions based on subjective experience and ‘feelings’ (i.e. one particular officer does not like a certain neighborhood  or perhaps is attempting to get more resources by overemphasizing trends – otherwise know as ‘agency interests’).
• Applying data analysis is a staged process whereby models are built, tested, and validated: Time series analysis can lead to observations such as seasonality (i.e. certain bars see more trouble during summer months).  By categorizing incidents (i.e. public disorderly, domestic disputes, drug possession), one can build an understanding of trends and consider amplifying factors (i.e. geographic location, economic trends, seasonality).  By subsequently re-applying statistical analysis to newly categorized phenomenon, interesting pattern insights can emerge (i.e. there is an increase in domestic incidents tied to drunk and disorderly incidents).
• More sophisticated quantitative methods add rigor to law enforcement strategy by basing decisions on evidence-based factors: Applying econometrics (regression analysis, examination of autocorrelation and lags) can add rigor to hypothesis regarding causation.  This approach, similar to how social scientists build a research hypothesis from simple statistics and substantiate with deeper analysis, allows law enforcement professionals to focus rigor in resource allocation that otherwise risks being misled by intuitive cognitive biases.
• It starts and ends with people: The station itself is a collection of people struggling to make sense of broader social trends and patterns.  For the application of a data analytics program to be effective, the systems involved in the analysis need to link cleanly with outlined processes (i.e. reviewing and discussing statistics at the start of each month when considering assignments and initiatives).
• Not all revelations are immediately actionable, but can be useful in orienting policy discussions: One initial observation I remembered was that nearly 50% of criminal incidents were linked to alcohol.  Another observation was that the only incidents related to marijuana use were crimes of possession.  However, this was largely outside our charter:  the station did not control liquor distribution directly, nor did it make the laws concerning marijuana.  However, armed with such statistics, the Police Chief was at least able to report to the town council on these facts and thus attempt to bring some logic and reason to political debates concerning local laws and policies (i.e. restricting opening times for bars, enforcing ‘blue laws’ concerning when alcohol could be sold, and, as applicable, overlooking minor marijuana infractions with deference to drunk and disorderly incidents).

What is new, in our brave new world of predictive policing?

• Sophisticated predictive analytics techniques are now embedded in software:  PredPol is a great example of a company that has applied advanced social science research to a software tool to improve law enforcement decision-making processes.  PredPol is based on research originating in complex algorithms used to predict earthquakes and aftershocks.  By using geographic information analysis and amplifying with predictive algorithms, PredPol gives police predictive cues concerning where subsequent outbreaks are more likely to occur.
• Pattern analysis has become more sophisticated:  Tools such as Viscovery allow raw data to be sifted in order to identify hidden quantitative patterns.  ‘Unsupervised’ techniques, for instance cluster and principal component analysis, allow law enforcement to identify natural categories resident in raw data.  For instance, it may appear that there are clusters of drunk and disorderly and domestic dispute incidents associated with a particular geographic area.  This knowledge can help to focus subsequent models and to suggest ‘experiments’.  For instance, the observation that a particular row of bars is the focus for crime incidents can lead to actions intending to reduce incidents, such as increased spot-checks, conversations with bar owners, and heightened street patrols.
• Machine learning paradigm shift:  Whereas traditional statistical analysis involves building and supporting a causal hypothesis (i.e. there is a seasonal and geographic aspect to increases in gun violence), machine learning focuses on observing correlative factors.  In a large dataset, machine learning can be used to generate predictions which do not necessarily care about implied causation as much as substantiating that a broad set of variables correlates reliably.  Google flu trends is a a good example: the platform predicts flu outbreaks based on correlation with a key set of trending search terms.  There is an ongoing debate and good reasons to be quite cautious with correlation-focused machine learning models (chiefly that they are subject to overfitting).  However, the utility and power of machine learning is clear as long as the models and techniques are applied by informed experts who are aware of the potential pitfalls and need for proper validation and testing.
• ‘Monolithic’ software packages have emerged:  SAS (Fraud framework), SAP (Fraud Management), Palantir, IBM (Analyst’s Notebook), and BAE Systems (Net Reveal) are examples of substantial information analytics ‘packages’ which can be deployed to drive a structured data analysis program.  An analytics program ideally drives analytics model creation such that the process of refining descriptive analytics into predictive insights and prescriptive actions becomes embedded in organizational processes.  Such packages are not simple nor inexpensive to deploy and manage, but evidence is that the efficiencies gained in terms of long-term cost savings and effectiveness can be worthwhile.
• Big Data and real time to the fore: Although some are convinced Big Data is a hype that has run its course, as per the Gartner technology hype cycle, the reality underlying the hype is now emerging into the mainstream.  In other words, Big Data is a reality.  What does this mean in practical terms?  Essentially, the means to store, transform, and analyze massive datasets are becoming increasingly available from an engineering standpoint.  There is an increasing trend to leveraging cloud-based storage and to implement real-time analytics decision making.  For policing and intelligence, this means being able to efficiently and quickly check across large, hybrid datasets in real time to identify individuals who may be wanted or connected to suspicious activities.  This also that ‘science fiction’ scenarios such as scanning license plates at a toll stop and instantly identifying owners with open arrest warrants is increasingly on the horizon (indeed may already be in place via certain authorities).

Since those days, now nearly 30 years ago, I can say that there are fundamentals which have not changed.  However, the scale and sophistication of solutions available to make law enforcement processes more efficient has evolved radically.  All that is required is developing the organizational will-power to adopt and implement evidence-based techniques.  This is the legacy of the Police Chief at my local station: leadership and structured management principles are the key to evolving from ad hoc approaches to evidence-based processes.  It is the foundation of moving from ‘making ends meet’ policing to embracing the principle of ‘serving and protecting’ as a trusted public steward.

For those interested to ‘go to the next level’ on this topic, the list of resources below may be of interest.

RELATED MEDIA

• aeon Precognitive police
• The Economist Don’t even think about it!
• sctr7 Excuse me, do you speak fraud?
• sctr7 Network analytics for fraud detection
• sctr7 Network analytics: more than pretty pictures
• sctr7 What information is gained from social network analysis?
• ACFE presentation advanced analytics for fraud detection and mitigation

LEARNING VIDEOS

• Network analytics for fraud detection and mitigation: part 1
• Network analytics for fraud detection and mitigation: part 2
• Overview and demonstration of semantic analytics (RSM Erasmus lecture)

ANALYTICS TOOLS

• Data mining / BI for crime: Sentient
• Predictive policing: PredPol
• Geographic information analysis: esri ArcGIS, CrimeStat
• Pattern / cluster analysis: Viscovery SOMine
• Social network analysis: SAS Infinite Insight, UCINET, Pajek, Gephi
• Text & sentiment analysis: SAS Text Miner
• Social media scanning ECM Universe
• Cognitive computing for discovery: IBM Watson

FRAUD, CRIME & INTELLIGENCE SOLUTION PLATFORMS

• BAE Systems Applied Intelligence
• Palintir Anti Fraud, Law Enforcement, Intelligence
• IBM i2 Analyst’s Notebook
• SAS Fraud Framework
• SAP Fraud Management

DATA STORAGE SOLUTIONS

• NOSQL graph databases: Neo4J, Titan, Oracle
• NOSQL document databases: MarkLogic

Analytics process