How Big Data boosts RegTech

1)   Banking: Regulatory Technology Basel III:

BASEL III norms are likely to be implemented in India from year 2019. This opens up a new paradigm in the field of machine learning and AI in terms of automating core tasks of not only assimilating transactions’ data in real time but also providing insights into data for better decision making and regulatory BASEL III reporting.

  1. With the compulsion to report various parameters at frequent intervals, banks will find it increasingly difficult to cope up with the pace and cost if they opt for traditional data collection & reporting technologies
  2. There will be a need to report on norms regarding Liquidity Coverage Ratio (LCR), Credit Value Adjustment (CVA), Statutory Liquidity Ratio (SLR), Cash Reserve Ratio (CRR), Credit Risk, Top 20 Counterparty report, Credit risks, etc. that would default to use of automation of data collection and analysis processes
  3. A major challenge would be collection of data, it’s secure transfer and automated intelligent interpretation to make decision making better

How Big Data boosts RegTech

Financial institutions’ KYC processes require information to be analyzed from both private and public sources in different languages and formats.

New data techniques mean that RegTech providers can aggregate data worldwide to make this process faster and more effective.

By gathering information from a wider variety of sources, including social media, Big Data can help to reveal hidden relationships.

Machine learning, which gives computers the ability to learn without being explicitly programmed, offers new opportunities to monitor for suspicious transactions across millions of accounts.


There are many uses of big data analytics by financial organisations to meet evolving regulatory and compliance requirements, Swift Institute reveals in its recent paper, “The Role of Big Data in Governance: a regulatory and legal perspective of analytics in global financial services”.

The paper cites two case studies as examples of how these analytical tools are influencing operational risk and practices within organisations across the financial services industry.

“With 2.5 quintillion bytes of data generated daily and regulators requesting more data from organisations, analytics have a big role to play,” says Peter Ware, director at Swift Institute. “The research dives into unchartered territory, highlighting the challenges and opportunities for financial organisations using business intelligence tools to improve operational efficiency and compliance.”

Highlights from the research include:

  • Organisations that harness the power of analytics to better understand organisational operations may reap many additional benefits beyond compliance;
  • Improved understanding of operational risks may also allow firms to reduce their requirements to hold higher levels of regulatory capital;
  • Analytics may help organisations better understand how individuals in the firm interact with one another and thereby act to improve lines of communication;
  • Analytics may also assist organisations in vital strategic decision making and related efforts to recruit and retain necessary staff;
  • Firms that embrace information governance techniques are better placed to exploit big data analytics and related future innovations.


Social media has opened new avenues and opportunities for organizations to connect with their customers, but the sheer volume of communications about brands, products and services; discussed, shared, criticised or liked on different social platforms can be overwhelming. Sentiment analytics helps to rapidly read all this data, providing an executive summary of what people like and don’t like about a company brand or products. The reasons behind the sentiment can then be easily extracted, providing valuable business insights.


Meaningful data includes opinions, feelings and attitudes about a brand, topic or keyword, which are shared freely in the world wide web. Knowing the opinion and attitude of the customers offers profound knowledge in order to adjust marketing tactics correctly.


The right sentiment analysis tool can also identify the most influential customers regarding company brands or products. It enables to engage with the right people who will spread the word and has much influence on a social platform. Exactly those key customers are critical in order to fulfil the goals for a successful acquisition strategy.


Many consumers freely give feedback and product suggestions on social media websites. Big Data technologies can be used to identify those valuable consumer insights to improve products and services much faster than with traditional surveys, which only portrays a small sample group at one specific moment in time.

An easy source of customer sentiment is from the social sphere, including social networks, blogs and review sites. This data is naturally unstructured and dynamic as new text is generated continuously. This data is then suited to measuring sentiment over time such as before and after an organizations branding efforts. Internal data gathered from past consumer surveys and call logs may also provide a good source to measure the customer sentiment towards particular products


Knowing the customer profiles means having a deep understanding about them, which can be used to drive actionable insights. This can lead to improved marketing campaigns, targeted sales and better customer service. A clearer view about the customer profile enables companies for example to send out triggered messaging, which is a good way to reinforce the brand and target customers.


Understanding how consumers are using a specific product and then making decisions accordingly makes a big difference. With the help of Big Data analytics companies can find out how engaged a consumer is with a product. This can help to send the correct marketing message and product when the customer needs or wants it most.


Analysing customer behaviour is not just reviewing past historical purchases, but a tool to forecast future actions and trends of customers. By predicting customer behaviour, insights can be revealed to stop churn before it is too late

Other relevant use cases in banking:


2) Telecom

Telecommunication companies can no longer afford to not make use of their big data. A simple Google search for “Telecommunications Big Data Analytics” turns up a measly 2 million+ results.

Analyze call detail records (CDRs)

Telcos perform forensics on dropped calls and poor sound quality, but call detail records flow in at a rate of millions per second. This high volume makes pattern recognition and root cause analysis difficult, and often those need to happen in real-time, with a customer waiting for answers. Delay causes attrition and harms servicing margins.
Hortonworks DataFlow (HDF™) can invest millions of CDRs per second into Hortonworks Data Platform, where Apache™ Storm or Apache Spark™ can process them in real-time to identify troubling patterns. HDP facilitates long-term data retention for root cause analysis, even years after the first issue. This CDR analysis can be used to continuously improve call quality, customer satisfaction and servicing margins.

Service equipment proactively

Transmission towers and their related connections form the spinal chord of a telecommunications network. Failure of a transmission tower can cause service degradation. Replacement of equipment is usually more expensive than repair. There exists an optimal schedule for maintenance: not too early, nor too late.

HDP stores unstructured, streaming, sensor data from the network. Telcos can derive optimal maintenance schedules by comparing real-time information with historical data. Machine learning algorithms can reduce both maintenance costs and service disruptions by fixing equipment before it breaks.

Rationalize infrastructure investments

Telecom marketing and capacity planning are correlated. Consumption of bandwidth and services can be out of sync with plans for new towers and transmission lines. This mismatch between infrastructure investments and the actual return on investment puts revenue at risk.

Network log data helps telcos understand service consumption in a particular state, county or neighbourhood. They can then analyse network loads more intelligently (with data stretching over longer periods of time) and plan infrastructure investments with more precision and confidence.

Recommend next product to buy (NPTB)

Telecom product portfolios are complex. Many cross-sell opportunities exist for the installed customer base, and sales associates use in-person or phone conversations to guess about NPTB recommendations, with little data to support their recommendations.

HDP gives a telco the ability to make confident NPTB recommendations, based on data from all of its customers. Confident NPTB recommendations empower sales associates (or self service) and improve customer interactions. An Apache Hadoop® data lake reduces sales friction and creates NPTB competitive advantage similar to Amazon’s advantage in eCommerce.

Allocate bandwidth in real time

Certain applications hog bandwidth and can reduce service quality for others accessing the network. Network administrators cannot foresee the launch of new hyper-popular apps that cause spikes in bandwidth consumption and then slow performance. Operators must respond to bandwidth spikes quickly, to reallocate resources and maintain SLAs.

Streaming data through HDF into HDP for real-time analysis can help network operators visualize spikes in call center data and nimbly throttle bandwidth. Text-based sentiment analysis on call center notes can also help understand how these spikes impact customer experience. This insight helps maintain service quality and customer satisfaction, and also informs strategic planning to build smarter networks.

Develop new products

Mobile devices produce huge amounts of data about how, why, when and where they are used. This data is extremely valuable for product managers, but its volume and variety make it difficult to ingest, store and analyze at scale. Not all data is stored for conversion into business insight. Even the data that is stored may not be retained for its entire useful life.
Apache Hadoop can put rich product-use data in the hands of product managers, which speeds product innovation. It can capture product insight specific to local geographies and customer segments. Immediate big data feedback on product launches allows PMs to rescue failures and maximize blockbusters.


Earth Observation and IoT Software-as-a-Service Big Data Opportunities Approach $2.4 Billion by 2026

Cambridge, MA – NSR’s Big Data via Satellite report, released today, finds over $580 Million in 2016 retail revenues attributed to Big Data via Satellite applications across seven key verticals, and by 2026 retail revenues will approach nearly $2.4 Billion.  Achieving that growth, however, will require on-going developments by satellite players to integrate, process, and analyze an ever-increasing range of data from both Earth Observation and M2M/IoT sources.


3) Healthcare

Innovation in healthcare technology started around 15 years back. One of the primary drivers of innovation, in the West, was the deployment of Electronic Medical Records (EMR) in hospitals and physician clinics. An electronic medical record is a term given to a medical record of patients collected in an electronic format. Before the advent of EMR, clinical information related to patients’ health was largely confined to paper, or written records. Therefore, EMR implementation in the West was the crucial push needed to get the innovation ball rolling in healthcare.
Despite its clear benefits, the real driver of EMR implementation for physicians in the western world was the regulatory pressure imposed by their governments.


On another note, a revolutionary change is taking place in the sphere of interconnected healthcare, which will have far reaching effects in the healthcare industry. Hospitals and physicians using EMRs are now beginning to understand the tremendous value in interconnecting patient information across their systems – using healthcare information exchange (HIE) technology. HIEs allow healthcare providers and patients to easily access and securely share a patient’s medical information electronically. With an increase in government regulations, US witnessed a significant growth in the HIE penetration over the last 10 years. Today, HIEs have evolved to showcase capabilities such as secure messaging, electronic record exchange, physician authentication, e-prescribing, event alerting and more. Among the many benefits of HIEs, one substantial value is that of standardization of data. Once data is standardized, it can be easily transferred to other systems and help improve patient care.

AI related technologies are disrupting healthcare sector
One of the pre-requisites of artificial intelligence is the availability of a lot of accurate patient information. A country like India faces one major challenge: currently there is no framework for hospitals and patients to implement an electronic medical record. So, the patient information is only available in a physical form, which doesn’t allow digital use of information to drive innovation.


Many industries use big data, but the healthcare industry is one of the vital areas where big data is profitably succeeding in shaping the usual practices.


Electronic health record is one of the widespread big data use cases in healthcare. Electronic health records keep track of each patient’s health chart and their medical reports, thereby reducing the need for duplicate tests and the associated cost.


Clinical support decision is a real-time application that can offer prescription after analyzing the medical data of a patient. This helps doctors analyze their patient’s health conditions and revert when necessary. Big data offers this crucial functionality where if a patient is suffering from, let’s say, blood pressure issues then a sudden increase or decrease of the same will be analyzed by their concerned doctor.


Evidence-based medicine aims at providing the doctors with the evidence of a patient’s record and compares the symptoms to a larger database of the patient; thereby enabling accurate, faster, and more efficient treatments. This big data use case helps in easy decision making.


Big data helps identify the at-risk patients based on their medical reports, records, and laboratory reports and offers them a reduced readmission rate. This allows a patient to focus more on her medical treatment and not on the readmission charges.


Every data is unique and it is essential to maintain data along with its security. This big data use case helps in dealing with fraudulence in the billing, personal identity, patient records. Insurance fraud has become a national problem where claimants try to obtain money that is not rightfully theirs. The insurance company also uses big data analytics to prevent the insurance claim frauds. Insurance data keeps on changing regularly, and hence it becomes necessary to maintain it regularly as well. Predictive analysis plays a vital role  in this process. Huge chunks of changing data are maintained and secured using this big data use case, with a result of meeting security constraints.

Big data analytics has become an important tool in today’s world since big data use cases are going wider than we thought. It helps firms, be it social media or healthcare, to reduce work to the minimum levels. Big data also helps in obtaining high productivity and growth, newer ideas, and in reducing time and cost involved.

Thus big data technologies are disrupting existing ways of doing business and are positively enabling newer analysis techniques. 


Leave a Reply

Your email address will not be published. Required fields are marked *