Adam Swartz

Storing user data required some lateral thinking. As users made changes to their CV the updates to CV components would be scored and added to the rest of the CV score. Scoring CV deltas saved processing time in the back-end AI model.

As the CV changes over time the user would soon become aware of the changes that brought positive benefit to their CV score.
Each individual score component was time stamped. Utilising Kimball Type 2 Dimension Methodology, the user's CV can be "rolled back" to any point in time.
Additionally, we could use the changes time series data to analyse user interaction with the application.

The database used is Postgres. Enhancements were made to the Sequelize Object Relational Model (ORM) node module to build and access the database model. These enhancements enabled inclusion of Type 2 Dimensionality to database tables.

Technologies: AWS, EC2, S3, Lambda, Aurora, SQL, DynamoDB NoSQL, Nodejs, Python, IAM, API Gateway, VPC, Cloudfront, ELB, Route 53

The challenge was to process one hundred million text documents (CVs and job descriptions) to build a corpus for a machine learning job to build a word embedding model (the AI).

The pipeline was designed to accommodate differing hardware demands of different layers of processing. Some processes required intensive disk access whereas other processes were RAM intensive. The solution included a dashboard for configuration and monitoring and a back-end controller service.

The Dashboard gave the user access to the controller. The user would define the processing code to be used in each processing layer and the number and type of compute resources for each layer. The user could then start, or stop and monitor running jobs.

The system would dynamically build layers of processing instances (EC2). A bespoke Linux (Ubuntu) EC2 image was built for this process. It was built to optimise certain processing requirements and include specific code libraries.

Each processing instance would use layer specific code written in Python, imported at runtime from our GitHub repository. Between each processing layer is a Kinesis Stream (AWS' Kibana implementation) to collate results and distribute to the next processing layer. I wrote the webapp and controller in node.js. The controller would orchestrate building and destroying of AWS resources via API calls utilising the node AWS SDK.

The Dashboard is an Express app. The controller is service written with Nodejs. The pipeline processing components are written in Python. Modules used include pandas, numpy and spacey.

Technologies: AWS, EC2, S3, Lambda, Kinesis, Aurora, SQL, DynamoDB, NoSQL, ECS, Nodejs, Python, Docker, ECR, Fargate

CaaS Short Selling service is a managed service for companies to track their exposure to short positions that might breach regulators' minimum allowed. The user delivers a positions file via FTP or email on a daily basis. For derivatives and indices we source reference data from across the internet. We break down the derivatives and indices to find the underlying equities.

The service uses AWS data pipeline service to process data. We use an AWS RDS (mySQL) database. Web crawlers scrape reference data from publically available sources. Each user is isolated to their own VPC. Reference data is stored in a central database accessible from each private network.

Technologies: AWS, EC2, Data pipeline, SQL, VPC, Cloudfront, ELB, Route 53

regulationnavigator.com is an Express web application that details regulations relevant the financial services industry.

It uses Firebase for user authentication and profile storage. The server is written in Nodejs with an Apache reverse proxy in front. The UI is bespoke to the project utilising bootstrap Material Design for the components. Data is loaded to the server via Zoho Creator application. The server runs on an AWS EC2 Linux instance.

The user can save and share reports and track compliance with regulations that impact their business.

Technologies: AWS, EC2, S3, Firebase, Nodejs

The FINREP and COREP projects both required extensive mapping of data from the company's Finance Data Warehouse to the reporting format required by the regulator. This mapping process was not a simple challenge.

I was given my own database with a snapshot copy of the Finance data warehouse. I had full control of the data. Working closely with Finance Change Business Analysts I developed a prototype to represent the data flow from the data warehouse to the reports.

Proxies needed to be found to split some data warehouse line items across particular line items in the reports. The code for this project was written in Oracle PL/SQL. I build the Business Analysts a mapping tool in Excel. They would list the mappings in the spreadsheet. VBA macros would source the data and process.

Once the prototype was complete, I oversaw the transition of the prototype into functional requirement documents for the developers to build this solution in the monthly processing run for the Financial Data Warehouse.

Large exposure to other financial sector companies was the cause of the Global Economic Crisis in 2007. Reducing the company's exposure to this risk was the remit of the FSE project.

Complex rules were defined by the regulator specifying how a company may offset long and short positions in other Financial Sector Entities. I was charged with developing a prototype to calculate the overall exposure to the finance sector.

Many positions were held in indices and derivatives. These needed to be broken apart to access the underlying shares and relative weightings. Rules to offset long positions with corresponding short positions were implemented. I used Oracle PL/SQL to build the prototype. Sourcing position data from the Financial Data Warehouse and splitting those position in derivatives to extract the Financial Sector Entities as defined by sourced reference data.

The data warehouse utilised Informatica for ETL storing data in a Sybase databse. In taking ownership of the warehouse I inherited a significant backlog of work including the integration of Basel II Liquidity Risk requirements.

The data model of the Data Warehouse needed to be changed. Additional data points needed to be stored against existing positions representing future cash flows. This required significant alterations to the Informatica ETL jobs. New data sources were added to the model to add new data.

Scaling up the team with staff from an external consultancy and internal hires I was able to manage the delivery of the change program on time and budget.

Incidentally, I was identified as a "key man risk" to the project and, on contract renewal, offered a completion bonus to stay to end of the contract.

Amongst other daily reports, determining on the sensitivity of bonds held to interest rate movements was an essential part of the Regulatory Reporting data Warehouse.

In the past, this calculation was performed by a batch script running on a dedicated pc under a desk. The code base, written in C++ used a generic Finance library to perform the calculation. This machine was a risk to the daily processing as it was not secure, not backed-up and subject to being accidentally un-plugged by the cleaners!

To bring the calculation inside the daily production process I wrote in Sybase T-SQL a Modified Duration calculator to perform this function. Using best practice interpolation technique to perform the calculation as per the bank's quantitative analysis department (Toronto) requirement.

The Market Risk System (MRS) used Value at Risk (VaR) to measure the sensitivity to interest rate movements.

The reporting layer of the MRS used Essbase as the OLAP engine to aggregate the equity risk across the organisational structure. VaR was the measure, however it cannot be aggregated. For reporting at each level in an organisation the overall position needed to be aggregated and then the VaR calculated at that level.

To solve this problem, and take advantage of some of the advanced Essbase functionality, I developed a dynamic calculation that would utilise pre-aggregated positions and apply the historical simulation calculation. The standard Essbase interpolation funcion did not perform as required by the user. Therefore I had to develop a new calculation to produce the correct result. This calcualtion was developed in java and wrapped in c++ to interface to Essbase.

The Essbase calcualtion engine aggregates calculation results up from leaf nodes to parent nodes up each dimension hierarchy. For positions held at parent level totals for child calculations over write the parent calculation result. Previous attempts to design the OLAP cube suffered from this problem. When I arrived on the team, I immediately identified this issue and altered the calculation code to correct for this error in design. Resulting is correct totals being aggregated throughout the OLAP model.