Devfest Cloud 2023
라이언의 꿀팁백과
1 <키노트> Building Your Tech Career with Developer Communities
스피커 : 개발자 생태계(DEVREL 데브렐) 팀 by Kristine Song (구글 8년차, 3번째 팀이고 이전에는 HR에 있었음)
1.1 Why communities matter?
- Learn
- Connect
- Inspire
최근에는 GitHub, Stackoverflow, 커뮤니티 기여 등 다양한 측면을 살펴봐서 채용을 진행함. 얼마나 배우려고 하는 열정이 있는지 등을 온라인 활동 등을 통해 살펴봄
1.2 Building your tech career - Growth opportunities
low hanging magic fruits : 튜토리얼 만들기, 오픈소스 기여, 행사 발표 등
high hanging magic fruits : 스타트업 창업 등
low hanging common fruits : 커뮤니티 참여, 책 읽기 등
많은 노력이 드는 데 효과가 없는 일은 할 필요 X. 여힝이나 가라.
Google Developer Experts 가 국내에 13명 있음
1.3 레퍼런스
- 행사 홈페이지 https://festa.io/events/4385
- Google Developer Groups https://developers.google.com/community/gdg
2 Build LLM-powered apps with embeddings, vector search and RAG
스피커 : Kaz Sato (Staff Developer Advocate (ML) @ Google Cloud (Tokyo)
Vertex AI Embeddings API + Vector Search - Grounding LLMs made easy
Integrating LLMs with existing IT systems without hallucination is crucial in business. LLMs are just probability language model; that's why there could be hallucination in their answers.
RAG는 LLM 과 Vertex AI 를 활용해서 할루시네이션을 없앰
What is Embeddings?
Embeddings are translating data into vectors. AI builds an embedding space as a map of meaning.
An embedding is a relatively low-dimensional space into which you can translate high-dimensional vectors. Embeddings make it easier to do machine learning on large inputs like sparse vectors representing words. Ideally, an embedding captures some of the semantics of the input by placing semantically similar inputs close together in the embedding space. An embedding can be learned and reused across models.
Vectex AI Embeddings API for Text
By calling the APIs provided by Google you can easily utilize embedding technologies.
ScaNN The vector search service for Google Searchv
2.1 RAG
Retrieval Augmented Generation is used to beat hallucination. by using vector search to support LLM.
2.2 VLM
LLMs with vision change businesses.
2.3 Reference
- https://ai-demos.dev/
- https://github.com/GoogleCloudPlatform/vertex-ai-samples/tree/main
- https://developers.google.com/machine-learning/crash-course/embeddings/video-lecture
- https://atlas.nomic.ai/map/vertex-stack-overflow
3 Your guide to tuning foundation models
스피커 : Erwin Huizenga (Staff Developer Advocate, Machine Learning Tech Lead)
파운데이션 모델은 사전에 엄청나게 많은 양의 데이터를 학습한 AI 모델을 의미
3.1 Why tuning?
원하는 목적에 최적화된 결과를 얻기 위해서. Take-specific tuning can make LMMs more reliable.
3.2 Limitations of prompt design
- Small changes in wording or word order can impact model results
- Inconsistent results
- Lots of trial and error
- Long prompts take up context window
It's non-deterministic!
Vertex AI tuning services
Here, we only cover two ways of tunings:
- Supervised Fine Tuning
- Reinforcement Learning from Human Feedback
3.3 Tuning foundation models
Full Fine Tuning : update all model weights
Adapter Tuning : update a small set of weights
3.4 Adapter tuning
- Train a small subset of parameters
- Parameters might be a subset of the existing model parameters or an entirely new set of parameters
3.4.1 Training benefits of adapter tuning
- Fewer parameters to update
- Faster training
- Reduction in memory usage
- Less training data needed
3.4.2 Serving benefits of Adapter Tuning
- Shared deployment of a foundation model
- 파운데이션 모델에 바꿔가면서 최적화 한 weights 을 손쉽게 적용 가능
- Augment with adapter weights specific to a particular task at runtime
3.5 Supervised Fine Tuning
Tuning with labeled dataset in the format:
{prompt, expected output}
3.6 RLHF
When the expected output is difficult to define
이거 두 개 읽어라
3.7 Gemini
Google's first foundation model that's built from the ground up to be multimodal.
- Gemini Ultra
- Gemini Pro
- Gemini Nano