Jihyung Ko고지형

Graduate Researcher — Computer Vision · Multimodal · Vision-Language Models

Seoul National University · AIBL Lab hanrista1157@snu.ac.kr
Jihyung Ko

Profile

Graduate researcher at Seoul National University's AIBL Lab (advisor: Prof. Kyungsu Kim), working on lightweight and reliable vision-language models for both general and medical domains. Prior industry experience as an AI engineer at Gauss Labs building semiconductor metrology and inspection systems. Interests span multimodal hallucination, model interpretability, computer vision, and end-to-end ML systems.

Education

2025 — present
Integrated M.S.–Ph.D. Program, Seoul National University
Interdisciplinary Program in Artificial Intelligence (IPAI) · AIBL Lab · Advisor: Prof. Kyungsu Kim
Research on lightweight general & medical vision-language models, multimodal hallucination, and interpretability.
2018 — 2023
B.S. in Computer Science & Engineering, Seoul National University
Best Undergraduate Thesis Award (2023)
Thesis: a GAN-based Korean CAPTCHA solver and an assessment of CAPTCHA vulnerability — GAN denoising followed by CNN character recognition to study which glyphs and noise patterns resist recognition.

Research Interests

Vision-language models & multimodal hallucination · model interpretability and training-free inference-time methods · medical & semiconductor vision · efficient / lightweight VLMs.

Publications & Preprints

* denotes equal contribution; † denotes corresponding author.

  1. PatchGate: Your VLM Already Sees the Objects It Forgets to Mention Under Review
    Jihyung Ko*, Eunji Jung*, Sanghyun Jo, Hyeongsub Kim, Ziseok Lee, Kyungsu Kim†
    ACL ARR 2026 (May) — under review · Co-first Author
  2. On the Collapse of Generative Paths: A Criterion and Correction for Diffusion Steering Accepted
    Ziseok Lee*†, Minyeong Hwang*, Wooyeol Lee, Sanghyun Jo, Jihyung Ko, Young Bin Park, Jae-Mun Choi, Eunho Yang†, Kyungsu Kim†
    ICML 2026
  3. SiliconBASE: Multi-task Baseline Model for Semiconductor Metrology and Inspection Applications Accepted
    Yonghyun Kim, Jihyung Ko, Sally Shin, Sang-Gil Park, Il Koo Kim
    SPIE Advanced Lithography + Patterning 2025 · Proc. SPIE 13426

Experience

2025.03 — present
Graduate Researcher — AIBL Lab, SNU IPAI
Advisor: Prof. Kyungsu Kim
  • Core research on lightweight foundational technology for general-purpose and medical vision-language models.
  • Training-free hallucination mitigation for VLMs via patch-level evidence projection.
2023.02 — present
AI Engineer (CVIP) — Gauss Labs
Full-time · Computer Vision & Image Processing
  • Built a Photoshop-style automation platform for semiconductor metrology, replacing manual code-writing by engineers; unsupervised object-definition model detects unseen objects and tolerates imprecise boundary input. Component- and semantic-object-based design; patent granted.
  • Improved an OOD-based defect anomaly detector that was unstable under slight misalignment; built an anomaly-score visualization & comparison app linked to fab data for per-process and intra-wafer analysis.
  • Led explainability work using large vision models to classify defect types from few examples (informed by a CVPR paper-reading group).
2021.12 — 2022.08
CVIP Intern — Gauss Labs (Seoul & US Office)
Applied Scientist → DevOps Engineer
  • Developed a recipe converter automating the data-processing pipeline; set up MLflow and improved a residual-CNN denoiser.
  • US office: built CI/CD pipelines on Azure DevOps and designed a microservice architecture independent of the legacy file system.
2021.06 — 2021.08
Vision AI Intern — SKT Vision AI Labs (T-WorX)
  • OCR detection & recognition of printed and handwritten text on labels/record sheets; benchmarked TextFuseNet against CRAFT and tuned an augmented MNIST pipeline.

Selected Projects

2021
AI Segmentation for Auto-Labeling (w/ Infinitt Healthcare)
Single-organ CT segmentation; compared and improved UNet3D, VNet, and ResNet-VAE.
2021
Background-People Removal in Crowded Scenes
Target detection + free-form image inpainting with gated CNNs; owned the inpainting and dataset-collection parts.
2020 — 2021
Wearable Chair
Sit/stand intent recognition (SVM) deployed on Raspberry Pi to auto fold/unfold a wearable chair for standing workers.
2019
SurBing — Full-stack Web Project
End-to-end build with React, Django, and NginX, from UI to server.

Activities & Awards

Activities

  • AttentionX 3rd cohort — AI research & startup group; plug-and-play LLM hallucination research (2024)
  • Waffle Studio — SNU web/app dev club; full-stack senior dev for SNU Calendar (2019–2020)
  • SNU Global Social Contribution Corps — rainwater purification install, Vietnam (2018–2019)
  • SNU VESS — fine-dust alert device "SEMI" (2018–2020)

Awards & Honors

  • Best Undergraduate Thesis Award, SNU CSE (2023)
  • Encouragement Award, SNU Social Contribution Contest
  • UCPC national programming contest — finals
  • Naver Boostcamp — Web Challenge completion (2020)

Technical Skills

PyTorch Vision-Language Models Object Detction/Segmentation Diffusion Models Interpretability Multi-GPU Training Python Django / FastAPI React C / C++ Go Java SQL Azure DevOps

Languages  Korean (native) · English (TOEIC 945)