Ginger Delmas

PoseEmbroider: Towards a 3D, Visual, Semantic-aware Human Pose Representation

Ginger Delmas, Philippe Weinzaepfel, Francesc Moreno-Noguer, Grégory Rogez

The PoseEmbroider framework combines multi-modal views of the human poses (images, textual descriptions, 3D joint rotations) to derive a rich visual-, semantic-, 3D-aware representation, which can be reused off-the-shelf in various downstream tasks (eg. any-to-any multi-modal retrieval, pose estimation, pose instruction generation). Its transformer core is trained in a contrastive fashion, and is shown to outperform the standard multi-modal alignment baseline.

PoseFix: Correcting 3D Human Poses with Natural Language

Ginger Delmas, Philippe Weinzaepfel, Francesc Moreno-Noguer, Grégory Rogez

We introduce the PoseFix dataset, which consists in over 6k triplets of 3D human pose pairs and a text modifier describing how the source pose needs to be modified to obtain the target pose. We further train a text-based pose editing model to generate corrected 3D body poses given a query pose and a text modifier; and a correctional text generation model, where correctional instructions are generated based on the differences between two body poses.

PoseScript: 3D Human Poses from Natural Language

Ginger Delmas, Philippe Weinzaepfel, Thomas Lucas, Francesc Moreno-Noguer, Grégory Rogez

We collect a dataset, PoseScript, pairing 3D human poses from AMASS and descriptions both written by human annotators and generated automatically by our proposed pipeline. We use PoseScript to train text-to-pose models, both for retrieval and generation. Pretraining on automatic data boost performance by a factor 2.

ARTEMIS: Attention-based Retrieval with Text-Explicit Matching and Implicit Similarity

Ginger Delmas, Rafael Sampaio De Rezende, Gabriela Csurka, Diane Larlus

We take inspiration from image retrieval and cross-modal retrieval to tackle the task of composed image retrieval: we design two complementary modules, each focusing on one modality of the query. The Explicit Matching module assesses how potential targets fit the textual modifier while the Implicit Similarity module compares potential target images to the reference image, assisted by the text. We validate our approach on FashionIQ, Shoes and CIRR.