facebook/timesformer-hr-finetuned-k600 is a forked repo from huggingface. License: "cc-by-nc-4-0"
Go to file
Niels Rogge 59b84d1f96 fix config files (#3)
- fix config file (26c03bd32420c58a81d1356b4cd7156f0889026f)
- Update preprocessor_config.json (d782cc491c1af6f559fed7e59d969ce499e04447)


Co-authored-by: Fatih <fcakyon@users.noreply.huggingface.co>
2022-12-12 12:53:13 +00:00
.gitattributes initial commit 2022-10-07 22:51:20 +00:00
README.md fix a typo in code snippet and processor config (#2) 2022-12-06 08:16:23 +00:00
config.json fix config files (#3) 2022-12-12 12:53:13 +00:00
preprocessor_config.json fix config files (#3) 2022-12-12 12:53:13 +00:00
pytorch_model.bin update weight and config to reflect latest changes from the pr (#1) 2022-12-02 08:13:46 +00:00

README.md

license tags
cc-by-nc-4.0
vision
video-classification

TimeSformer (base-sized model, fine-tuned on Kinetics-600)

TimeSformer model pre-trained on Kinetics-600. It was introduced in the paper TimeSformer: Is Space-Time Attention All You Need for Video Understanding? by Tong et al. and first released in this repository.

Disclaimer: The team releasing TimeSformer did not write a model card for this model so this model card has been written by fcakyon.

Intended uses & limitations

You can use the raw model for video classification into one of the 600 possible Kinetics-600 labels.

How to use

Here is how to use this model to classify a video:

from transformers import AutoImageProcessor, TimesformerForVideoClassification
import numpy as np
import torch

video = list(np.random.randn(16, 3, 448, 448))

processor = AutoImageProcessor.from_pretrained("facebook/timesformer-hr-finetuned-k600")
model = TimesformerForVideoClassification.from_pretrained("facebook/timesformer-hr-finetuned-k600")

inputs = processor(images=video, return_tensors="pt")

with torch.no_grad():
  outputs = model(**inputs)
  logits = outputs.logits

predicted_class_idx = logits.argmax(-1).item()
print("Predicted class:", model.config.id2label[predicted_class_idx])

For more code examples, we refer to the documentation.

BibTeX entry and citation info

@inproceedings{bertasius2021space,
  title={Is Space-Time Attention All You Need for Video Understanding?},
  author={Bertasius, Gedas and Wang, Heng and Torresani, Lorenzo},
  booktitle={International Conference on Machine Learning},
  pages={813--824},
  year={2021},
  organization={PMLR}
}