Introduce
As a Software and Data Engineer, I possess expert knowledge and experience in designing, developing, and operating enterprise backend servers and data platforms. Additionally, as a full-stack and full-cycle engineer, I have a interest in frontend development, as well as a passion for building CI/CD pipelines and fostering a DevOps culture. This comprehensive skill set allows me to contribute effectively across the entire software development lifecycle, from conception to deployment and maintenance.
Skills
- Software Engineer | CI/CD | DevOps | DataOps | Data Engineer | | Data Platform Engineering | Full-Stack
- Full-Cycle | Backend API | Python | Serverless | AWS | Microsoft Fabric | Cloud | Code Architect | Object-Oriented Programming
- Clean Architecture | Microservice | React | React Native | JavaScript | TypeScript
Experience
Enterprise Data Lake & CI/CD Design:
- Led and designed CI/CD pipelines for a stable and scalable enterprise data lake on Microsoft Fabric.
- Transitioned from an on-premises pipeline architecture to a cloud-first approach, enabling automated deployment and updates.
- Developed a No-code platform for pipeline creation, empowering non-technical users to design and deploy data pipelines.
- Skills: Azure DevOps, Microsoft Fabric, Airflow
Middleware Server Development:
- Developed on-premises middleware servers that connected Microsoft Fabric with external storage, ensuring seamless communication between environments.
- Skills: FastAPI
Observability:
- Built a monitoring system, improving fault detection across pipelines using ElasticSearch and Kibana dashboards.
- Skills: ElasticSearch, Kibana
DevOps Culture Establishment:
- Advocated and implemented a DevOps culture, including the introduction of agile methodologies, standardized code reviews, JIRA workflows, and detailed documentation.
System Redesign and AWS Migration:
- Led and designed a backend system in an AWS Serverless environment using Python and Clean Architecture to replace a legacy system.
- Enhanced system performance by redesigning backend infrastructure, code, and CI/CD pipelines.
- Built CI/CD pipelines using AWS CodePipeline, CodeBuild, and CodeCommit for automated deployment and testing.
- Developed monitoring and alert systems using AWS QuickSight dashboards and Slack API integrations to track key metrics.
- Migrated all user operations from the legacy system to the newly designed serverless architecture, reducing costs and increasing reliability.
- Skills: AWS Lambda, API Gateway, AWS CodePipeline/CodeBuild/CodeCommit, DynamoDB, S3, QuickSight, CloudWatch
DevOps Culture Establishment:
- Introduced a culture of continuous deployment and Agile methodologies using tools like Confluence for documentation, JIRA for task management, and Git for collaborative development (pull requests, commit rules).
- Advocated and implemented a DevOps culture, including the introduction of agile methodologies, standardized code reviews, JIRA workflows, and detailed documentation.
Data Pipeline Development for PII Compliance:
- Developed pipelines to classify and process large volumes of data in compliance with PII (Personal Identifiable Information) regulations.
- Skills: AWS Glue, EMR, Athena, Spark
Middleware Server Development for LG TV and ThinQ API Integration:
- Developed and operated middleware servers enabling communication between LG Smart TVs and ThinQ API.
- Facilitated the transmission of control/status requests from TV users to ThinQ API and delivered appliance notifications to TV users.
- Improved system performance by removing an unnecessary processing hop, reducing both costs and latency.
- Skills: API Gateway, Lambda, CloudWatch, AWS IoT, DynamoDB
ThinQ API Server Development:
- Designed and implemented APIs to allow external services to control and monitor LG appliances.
- Adopted a microservices-based architecture to enhance scalability and reusability.
- Improved query performance by redesigning the DynamoDB schema and reducing unnecessary Lambda calls to optimize costs and performance.
- Skills: API Gateway, Lambda, CloudWatch, DynamoDB, S3, ElastiCache, SQS, SNS
CI/CD:
- Built a robust CI/CD environment enabling fast and iterative development cycles with weekly deployments.
- Initially implemented using CloudFormation, later transitioned to Serverless Framework to streamline serverless resource management.
- Migrated to AWS CDK for improved resource management and code reusability beyond serverless configurations.
- Skills: Jenkins, AWS CDK, Serverless Framework, CloudFormation
Data Processing Pipeline Development for ThinQ API Server:
- Built scalable pipelines to process and transform raw logs and database data into structured metrics.
- Extracted and structured over 400GB of daily log data to generate operational insights and development metrics and visualized key metrics using AWS QuickSight.
- Skills: CloudWatch (EventBridge), Lambda, DynamoDB, S3, Step Functions, QuickSight
Education
- Major in Computer Science
- Thesis: Cache Replacement and Data Placement for Multi-Cloud Online Social Network Services
- Major in Computer Engineering
Projects
- Created a web service analyzing user-uploaded photos to match them with SBS TV show contestants, suggesting the most compatible matches.
Publications
- Book (In Progress): Clean Architecture with Python. (2024 – Ongoing)
- Patent: AI patent of a practical technique to enhance accuracy in multiple classification scenarios. (2022)
- Journals: Adaptive Data Placement for Improving Performance of Online Social Network Services in a Multicloud Environment (2017)
- Conferences: A Social-Aware Caching Algorithm for Improving Performance of Online Social Network Services in a Multi-Cloud Environment (2017)
Others
- Award: Excellent Oral Presentation Certificate: Awarded at the 2017 the 3rd International Conference on Communication and Information Processing.
- Certification: LG Electronics SW Test: Perfect Score (2021)
Copyright © 2024 Alphahacker. All rights reserved.