Data Acceleration
₹750-1250 INR (1시간 기준)
Health Care Service Corporation (HCSC) is a not-for-profit corporation health insurance company in the United States. The current scope of Data Accelerator project is data ingestion and data processing. In this project we are ingesting data from different data sources to the data lake by applying the required business transformation rules and later analyzing the data for faulty records.
This project was developed using Agile development methodology having Sprint of 3 weeks.
Role and Responsibilities:
As part of each Sprint, we were allocated respective target tables to which data should be loaded.
Fixed length/Delimited flat files were loaded into the Source tables.
According to the Mapping document we design the join criteria for the source tables.
One or more source tables to be joined, applying different business rules and loading to Temp table.
Writing Scala code to perform transformations joins on data frames and removing duplicates before loading to the target table.
Developing code involved writing shell script, HQL’s, Spark-Scala code.
Zena is used for job scheduling.
프로젝트 ID: #26375393
프로젝트 소개
이 일자리에 대한 프리랜서 2 명의 평균 입찰가: ₹1225 (1시간 기준)
Hi, I am a bigdata developer and a module lead in reputed MNC.i an into the IT industry for more then 12 years. I have tonnes of experience in developing projects using Java,Apache Spark,Hive,Kafka,Scoop,Pig,Scala,aws 기타
Hi, I have 10 years of IT experience and I have worked on SQL, Python, Shell scripting, Bigdata(Hadoop), Spark(Scala), Machine Learning models, NLP, Classification and Regression models. Since the last 7 years I am w 기타