Lecture 16: ICLR conference 2013 highlights

What is ICLR

ICLR -- International Conference on Learning Representations, one of the most important A* conferences.

Started in 2013.

Numbers of 2023: 4,966 total submissions, 1574 accepted (32%).

Statistics: https://guoqiangwei.xyz/iclr2023_stats/iclr2023_submissions.html

Submission

Paper submission deadline: 21 September 2022.

Reviews released: 5 November 2022

Acceptance notification: 21 January 2023

Each paper receives at least 3 reviews, then Area Chair writes 'meta-review', Senior Area Chair confirms it and finally --- Program Chairs release the decision.

Venue

First time in Africa. Should have been in Addis-Abbaba in 2020, got cancelled.

Now it was in Kigali, Rwanda.

First offline ICLR conference since pandemic!

The location was a modern convention center.

How things work

It is full-week intensive work.

There are talks, but most interesting (besides networking) are poster sessions.

Invited talk

An interesting invited talk by Sofio Crespo

on bio-inspired art.

Funny quote:

Interesting (to me) posters

Summarization programs

Convolutional neural operators

S5

SSM models are quite popular:

Some paper are quite strange

A little bit of work on certification and robustness (but not much)

Quite a few papers on drug design

Absolute gem: designing matrix floating point units

Several attempts to 'marry' graphs and transformers

Learning permutations in a differentiable way

Several approaches to discrete diffusion (note drug application)

Several papers on AI&Physics

This papers learns the system from trajectories with learned invariants as well.

Hungry Hungry Hippos

Potentially, the new milestone for sequence modelling, H3 model.

New: journal track

More theoretical work has been presented.

Anchors to represent the point clouds

Simple idea (could be improved by NLA in my opinion)

Subset selection for active learning

A lot (!) of prompt-based research, for example, Flip the instruction

Idea: try to make artificial data by learning the instruction from the result.

I learned about new stuff (Ramanujan graph)

This work has been related to lottery ticket hypothesis. Sparsity of learned representation has been the subject of several talks.

DiffDock: SOTA for docking

Prompting for better math proofs

Drug-design with transformers

Multi-agent system, tries to learn a language (Hmmm)

Trained transformers are sparse

Differentiating through (some) combinatorial solvers

Measuring 'rarity' of the samples

Programming better (cool work!)

Termination problem in NLP

Making deep graph network models more stable to depth

Hyperbolic RL