Jaywing won the 2018 Credit & Collections Technology Award for Technology Team of the year. This review takes a look at Jaywing’s work and the criteria its team met to win the award.
Category Title: Technology Team of the Year
Award winner: Jaywing
Title of winning projects/initiative: Product development team, Archetype
Key team members: Martin Benson, Martin Smith, Nick Priestley, Nathaneal Panter, Dan Martin, Steve Lewis
Entry criteria: This award recognised a team who has exceeded in technological innovation to provide compliant AI-powered risk models to the industry.E
Jaywing’s basis for entry
Our team is at the forefront of AI and data science, driving value from data and developing relevant technology that directly grows business opportunities for both Jaywing and its clients. Entering the Technology Team of the Year award at this year’s Credit & Collections Awards gives our team recognition within the credit risk sector for the development of such a unique and innovative product.
What was the scale of the projects your team faced?
Jaywing’s Product Development team created pioneering AI-based predictive modelling technology, Archetype. Building on a previous development of a machine learning platform to deliver compliant IFRS9 models, the team turned their attention to solving the problems associated with the use of AI in predictive modelling for credit, fraud and marketing.
With scepticism around the use of AI in creating credit risk models due to a lack of transparency, the team was faced with the challenge of developing a solution that would allow lenders to create explainable and transparent models to cater for the needs of regulators and internal stakeholders, enabling clients to understand their models and ensure they meet internal business rules. Archetype is unique because it solves the ‘black box’ problem around explainability in the creation of models using AI.
Who was the target group?
Archetype uses AI to transform the way in which models are generated and is a genuine game changer for credit scoring and assessment. It allows lenders to control and explain a neural network model and enables them to use AI in credit and fraud risk assessments where it was previously difficult to do so.
What was the approach?
The team’s remit is one of research and development, bringing to market innovative products which solve a client problem in a way which isn’t catered for by any existing solutions.
In this case, the team started with an intense period of research into recent mathematical developments that have enabled AI models to generalise better, ultimately inventing a unique mathematical approach which enables Archetype to specify direct links between input variables and the model output – something which was previously impossible to achieve. Jaywing has applied for a patent for this mathematical breakthrough.
Having solved the mathematics, the team then built the approach into an intuitive, browser-based software application which automates previously time-consuming aspects of modelling, using AI techniques to replace the model production. This frees analysts to focus on optimising the use of their data within the model, squeezing more insight from their data and enabling the introduction of new data sets to drive further value.
What were the Key Performance Indicators (KPIs)?
The KPIs for this project relate to a number of achieved outcomes:
- Generate more predictive models than are possible through traditional techniques
- Retain an intuitive link between input variables and the resulting prediction
- Enable the analyst to understand and control the models that they generate
- Make it easy to model, deploy and execute the resulting models.
Archetype is currently the only commercially available Neural network-based scoring product which solves the ‘black box’ problem around explainability in the creation of models using AI.
It overcomes this problem by enabling the analyst to specify up-front how each variable should behave within the model, giving confidence that the model is not over-fitted and that decisions are robust.
This ensures that model governance continues to play a leading role in how models are developed, putting it as an up-front step in the modelling process.
It is also able to demonstrate the impact of changes in each individual variable across a range of data points, giving confidence that any constraints are correctly applied and enabling the analyst to understand the overall shape or behaviour of unconstrained variables within the model.
How was teamwork demonstrated?
Utilising expertise and specialist AI and machine learning know-how, the team was able to launch a ground-breaking product.
With both the breadth of experience and the wider-business’ award-winning digital agency (Epiphany), recognised within the digital sector for its cutting-edge design and development work, the team was able to develop and test methodologies and frameworks, to create a product which meets regulatory requirements and is robust, simple and easy to use.
What impact has the winning team had in working with its clients?
Hitachi Capital Consumer Finance wanted to understand the potential to significantly improve its existing application credit scores using Jaywing’s AI modelling technology, Archetype.
Using Archetype to replace traditional modelling steps, Jaywing was able to apply appropriate constraints which reflected how Hitachi’s modellers would apply common sense rules to the inputs. These rules meant that the system would only generate model outputs adhering to Hitachi’s expectations, making models explainable to regulators and customers alike – a key differentiator compared to other Neural Network-based approaches. Most importantly, the advanced modelling tool demonstrated impressive uplifts, saving lenders potentially millions!
Archetype demonstrated an impressive uplift of between 7.2% and 11% for Hitachi Capital’s portfolios. Other clients have subsequently seen similar or even greater uplifts, with a corresponding impact on bad debt.
Winning this award has generated fresh interest in Archetype and has opened up conversations with a number of clients for the first time, helping them to realise the potential of AI within their business.
Who was involved?
Martin Benson, Jaywing’s Head of AI and Martin Smith, Jaywing’s Head of Product Development, together with senior mathematicians and data and computer scientists were involved in the creation of Archetype. The team have, on average, 18 years’ deep experience of the credit industry, having built long-standing (10+ years) relationships at Jaywing and working with the likes of Lloyds Banking Group, Royal Bank of Scotland, Nationwide, Secure Trust Bank, Shawbrook Bank, Paragon Bank and a number of leading building societies. The team worked closely with an award-winning team of front-end developers from Jaywing’s sister company, Epiphany.
Were there any hurdles the team faced along the way?
Until recently, many barriers have been in place preventing the full adoption of AI in Risk and preventing the huge uplifts in predictive power that such approaches can generate. The barriers were, in part, due to the need for credit teams to understand the decisions that their scores and models were making.
With traditional credit scores, there is a clear link between the input variable and the effect that a change in the variable will have on the resulting score. When using neural networks, the non-linearity of models means that understanding these models is much more difficult to achieve. Archetype is currently the only commercially available Neural Network-based scoring product which solves the ‘black box’ problem around explainability in the creation of models using AI and, unlike other approaches, offers a mathematical guarantee that its output will adhere to behaviours specified by the user for each individual variable.
Solving the black box problem is important because a lender needs to explain their decision to three groups. To customers: why have I been rejected? To the regulator: how robust are your credit decisions? To internal credit policy functions: are we accepting and rejecting the right people?
What the Judges said:
“Both the external recognition of the expertise in the team and the development of innovative products are impressive achievements. Archetype in particular may prove to be an important innovation in the sector. Being able to bring these products to market in often conservative client cultures proves the robustness of the arguments and evidence.”
“Great and unique modelling solution for credit scoring and assessment – using AI to transform the way in which models generate Risk. A team of data scientists that specialise in AI, having successfully launched several innovative projects.”
Jaywing’s Ben O’Brien, Managing Director said “Congratulations to all the Jaywingers involved in the creation of Archetype – a worthy win. Our team have many years’ deep experience in the credit industry and Archetype leads the field in enabling lenders of all sizes make use of this transformative technology and it’s a genuine game changer for credit scoring and assessment. This award recognises that our team is at the forefront of AI and data science, driving business value from data and developing relevant technology that directly grows business opportunities for both Jaywing and its clients.”
The 2019 Credit & Collections Technology Awards opens for entries on 31st January 2019, for information on how to ente or to review sponsorship options contact: firstname.lastname@example.org