Urgent

Software Engineer - LLM Pre-Training

Preferred Networks, Inc.

Tokyo
101 to 1,000
Company Homepage

Job Summary

Preferred Networksでは大規模言語モデル(LLM)を中心としたマルチモーダルな基盤モデルの開発を進めています。LLM・生成AIの基盤モデル開発に加え、エンタメ、科学計算、ロボットといった自社事業領域を強化するための応用開発や、多様な産業への応用研究も行っています。事前学習チームの役割は、LLMの大規模な事前学習を行い、高性能な事前学習モデルを構築することです。事後学習を終えた後のモデルが実用的で価値の高いものとなるようにLLMに知識や能力を学習させる必要があります。強化学習などの手法の発展により、LLMは事後学習によって様々な能力を発揮させることができるようになっている一方で、事前学習で全く学習できていない能力を事後学習で発揮させることは難しいという報告もあります。事後学習で高い性能を発揮させるためには、事前学習での取り組みも重要です。大規模言語モデル - 事前学習エンジニアは、高性能なLLMの事前学習のために必要な研究開発に取り組みます。事前学習モデルの最終的な性能を決めるデータセット戦略・モデルアーキテクチャ・学習手法の研究開発以外にも、大規模・長期間の学習を効率よく安定して実行し続けるための取り組みなども行っています。 Preferred Networks is developing multimodal foundation models centered around Large Language Models (LLMs) and is expanding its research into applied development and industrial application research to strengthen its business domains in entertainment, scientific computing, robotics, and others. The pre-training team’s role is to conduct large-scale pre-training of LLMs and develop high-performance pre-trained LLMs. The goal is to equip the LLM with the necessary knowledge and capabilities to ensure that models after post-training are practical and valuable. While advancements in techniques like reinforcement learning have enabled LLMs to demonstrate various capabilities through post-training, some reports suggest that achieving capabilities that were not learned during pre-training through post-training alone can be challenging. To achieve strong performance in post-training, pre-training efforts are equally crucial. As a LLM Pre-training Engineer, you will engage in research and development required for high-performance LLM pre-training. In addition to developing datasets strategy, model architectures, and training methodologies that ultimately determine the final performance of pre-trained models, you may also work on optimizing and ensuring efficient, stable execution of large-scale, long-term training.

Company Info

Preferred Networks, Inc. (PFN) is a Japanese company that vertically integrates the AI value chain, from AI chips and computing infrastructure to generative AI, solutions, and products. They focus on applying cutting-edge machine learning and deep learning technologies to real-world applications in various industries, including transportation systems, manufacturing, bio-healthcare, and IoT.

Login to see all information about offers.

By registering with LabBase Plus you...

You can select from our vast stock of job offers.

Can view all curation offers.
Receive notifications of new offers added to curations.
Possibly be scouted by corporations.
Search for companies with unique technologies.
Search for job offers in areas like inorganic chemistry material science.

Registration takes roughly 2 minutes