Lecture 16: ICLR conference 2013 highlights¶

What is ICLR¶

ICLR -- International Conference on Learning Representations, one of the most important A* conferences.

Started in 2013.

Numbers of 2023: 4,966 total submissions, 1574 accepted (32%).

Statistics: https://guoqiangwei.xyz/iclr2023_stats/iclr2023_submissions.html

Submission¶

Paper submission deadline: 21 September 2022.

Reviews released: 5 November 2022

Acceptance notification: 21 January 2023

Each paper receives at least 3 reviews, then Area Chair writes 'meta-review', Senior Area Chair confirms it and finally --- Program Chairs release the decision.

Venue¶

First time in Africa. Should have been in Addis-Abbaba in 2020, got cancelled.

Now it was in Kigali, Rwanda.

First offline ICLR conference since pandemic!

The location was a modern convention center.

How things work¶

It is full-week intensive work.

There are talks, but most interesting (besides networking) are poster sessions.

Invited talk¶

An interesting invited talk by Sofio Crespo

on bio-inspired art.

Funny quote:

Interesting (to me) posters¶

Summarization programs

Convolutional neural operators¶

S5¶

SSM models are quite popular:

Some paper are quite strange¶

A little bit of work on certification and robustness (but not much)¶

Quite a few papers on drug design¶

Absolute gem: designing matrix floating point units¶

Several attempts to 'marry' graphs and transformers¶

Learning permutations in a differentiable way¶

Several approaches to discrete diffusion (note drug application)¶

Several papers on AI&Physics¶

This papers learns the system from trajectories with learned invariants as well.

Hungry Hungry Hippos¶

Potentially, the new milestone for sequence modelling, H3 model.

New: journal track¶

More theoretical work has been presented.

Anchors to represent the point clouds¶

Simple idea (could be improved by NLA in my opinion)

Subset selection for active learning¶

A lot (!) of prompt-based research, for example, Flip the instruction¶

Idea: try to make artificial data by learning the instruction from the result.

I learned about new stuff (Ramanujan graph)¶

This work has been related to lottery ticket hypothesis. Sparsity of learned representation has been the subject of several talks.

DiffDock: SOTA for docking¶

Prompting for better math proofs¶

Drug-design with transformers¶

Multi-agent system, tries to learn a language (Hmmm)¶

Trained transformers are sparse¶

Differentiating through (some) combinatorial solvers¶

Measuring 'rarity' of the samples¶

Programming better (cool work!)¶

Termination problem in NLP¶

Making deep graph network models more stable to depth¶

Hyperbolic RL¶

In [ ]: