Building a multidisciplinary granular data space (DataLake)

CLIENT NAME
Central bank of European Country
SECTOR
Financial
MISSION
Building a multidisciplinary granular data space (Datalake) that provides a flexible and innovative service to internal users.

ABOUT THE CLIENT

The Client is a state bank of one of European countries. The bank is responsible for the accounts of the government, managing the accounts and the facilitation of payments for the Treasury and some public companies. It also oversees the auctions of public securities on behalf of the European Central Bank.

KEY ACHIEVEMENTS

  • Building a DataLake to store large volumes of data from multiple sources and in various formats providing a flexible and innovative service to internal users.

THE CHALLENGE

  • Facing the prospect of massive, granular data, statistical processing that must be adapted to increase data analysis capabilities and employee productivity
  • Providing a storage for Build a DataLake to store large volumes of data from multiple sources and in various formats, for example, ANACREDIT: monthly collection of 94 attributes for 21 million lines of credit

THE SOLUTION

The success of the project has been in the development of roadmap to facilitate principles to build a stable, large data space system covering the creation of the DataLake base and technical implementation.

2017

  • Phase 1 of development of the DataLake base in agile method, 3-week sprints
  • Development of the first ANACREDIT, BASTID and DATAGAPS client projects in plateau mode in the premises of the Banque de France

2018

Production of the DataLake technical base

Phase 2 of development of the DataLake base and production of customer projects:

  • DATAGAPS (T1)
  • BASTID/EMBARGO (T2)
  • ANACREDIT (T3)
  • BASTID/EVOLMPM (T4)

THE TECHNOLOGIES

  • Hadoop
  • HDFS, Hive, Hbase
  • Oozie
  • Atlas
  • Ranger, Knox
  • ElasticSearch, Kibana
  • WildFly
  • PostgreSQL
  • Java, R
  • Gitlab, Jenkins

THE RESOURCES

An Agile team of :

  • 1 Project Director
  • 1 Project Manager / Scrum Master
  • 3 Technical leads
  • 4 core developers
  • 5 ANACREDIT developers
  • 1 DATAGAPS developer
  • 3 BASTID developers

THE RESULTS

  • Supported the evolution of user usage
  • Provided an agile and innovative services to customers
  • Facilitated collection and analyse of a large amount of data
  • Implemented modern reporting tools to BDF agents
  • delivered simplified data sharing tool
  • Increased data analysis capabilities and the productivity of work
  • Contributed to the digital transformation of the Client

EXPERTS SAYS :

“Technical expertise is an essential element in supporting customers in their digital transition”.
-Infotel Project manager