This talk will present exploratory work using self-supervised representation learning on time-domain astronomy data. Recent advances in deep learning, notably for Natural Language Processing tasks, have highlighted the ability of DL methods to handle sequential data. Our goal is to explore the properties of the representation-space learned by these methods, and if this space is structured or informative about e.g. the morphology, the behaviour or the underlying properties of the observed objects. These methods could then be useful tools to perform class discovery (e.g. by clustering similar behaviours together) and/or anomaly-detection (‘global’ and ‘local’) by working directly on the representation-space.