Seminarium AI Bay 23.03.2023 | FTiMS - Politechnika Gdańska

Polskie Stowarzyszenie Sztucznej Inteligencji, przy współpracy z AI Bay, zaprasza na seminarium naukowe organizowane w formie zdalnej 23.03.2023 o godzinie 17.00.

Swoją prezentację przedstawi dr Stanisław Jastrzębski z firmy Molecule.one. Poniżej treść wystąpienia i bio kandydata.

 

Abstract: The early phase of training of deep neural networks holds many mysteries. For example, using a large learning rate in the early phase of training is critical for achieving good final performance of the model. I will describe our work on understanding these effects. First, I will describe how the simplicity bias emerges in the beginning of training. Second, I will propose that the mechanism by which using a large learning rate improves generalization is by regularizing the local curvature from the outset of training. This mechanism is connected to the fact that the early learning dynamics are chaotic due to large local curvature. To corroborate this mechanism, we have designed an explicit regularizer that enables training well-generalizing networks using a small learning rate.

 

Bio: Stanislaw Jastrzebski serves as the CTO and Chief Scientist at Molecule.one, a startup speeding up drug discovery by making organic chemistry more predictable. He is passionate about improving the fundamental aspects of deep learning and applying it to automate scientific discovery. He completed his postdoctoral training at New York University. His PhD thesis was based on work on foundations of deep learning done during research visits at MILA (with Yoshua Bengio) and the University of Edinburgh (with Amos Storkey). He received his PhD from Jagiellonian University, advised by Jacek Tabor. He gained industrial experience at Google, Microsoft and Palantir. He has published at leading ML venues (NeurIPS, ICLR, ICML, JMLR). He is also actively contributing to the machine learning community as an Area Chair (most recently for NeurIPS 2023) and as an Action Editor for TMLR.

W trakcie spotkania przewidywana jest żywiołowa dyskusja.

Spotkanie będzie zorganizowane za pośrednictwem platformy Zoom. Prosimy o rejestrację w celu uzyskania linku do spotkania (zostanie przesłany najpóźniej dzień przed spotkaniem).

Rejestracja: https://aibay.ai/seminarium-polskiego-stowarzyszenia-sztucznej-inteligencji-oraz-ai-bay-23-03-2023/

Serdecznie zapraszamy
Patryk Jasik