facebook
/
detr-resnet-50-panoptic
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Improve README

This commit is contained in:
Niels Rogge 2022-08-05 13:29:29 +00:00 committed by huggingface-web
parent fc15262cfd
commit 060ed34a4a
1 changed files with 25 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

34
README.md
View File

@ -30,6 +30,8 @@ The model is trained using a "bipartite matching loss": one compares the predict
DETR can be naturally extended to perform panoptic segmentation, by adding a mask head on top of the decoder outputs. DETR can be naturally extended to perform panoptic segmentation, by adding a mask head on top of the decoder outputs.
![model image](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/model_doc/detr_architecture.png)
## Intended uses & limitations ## Intended uses & limitations
You can use the raw model for panoptic segmentation. See the [model hub](https://huggingface.co/models?search=facebook/detr) to look for all available DETR models. You can use the raw model for panoptic segmentation. See the [model hub](https://huggingface.co/models?search=facebook/detr) to look for all available DETR models.
@ -39,22 +41,36 @@ You can use the raw model for panoptic segmentation. See the [model hub](https:/
Here is how to use this model: Here is how to use this model:
```python ```python
from transformers import DetrFeatureExtractor, DetrForSegmentation import io
from PIL import Image
import requests import requests
from PIL import Image
import torch
import numpy
url = 'http://images.cocodataset.org/val2017/000000039769.jpg' from transformers import DetrFeatureExtractor, DetrForSegmentation
from transformers.models.detr.feature_extraction_detr import rgb_to_id
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw) image = Image.open(requests.get(url, stream=True).raw)
feature_extractor = DetrFeatureExtractor.from_pretrained('facebook/detr-resnet-50-panoptic') feature_extractor = DetrFeatureExtractor.from_pretrained("facebook/detr-resnet-50-panoptic")
model = DetrForSegmentation.from_pretrained('facebook/detr-resnet-50-panoptic') model = DetrForSegmentation.from_pretrained("facebook/detr-resnet-50-panoptic")
# prepare image for the model
inputs = feature_extractor(images=image, return_tensors="pt") inputs = feature_extractor(images=image, return_tensors="pt")
# forward pass
outputs = model(**inputs) outputs = model(**inputs)
# model predicts COCO classes, bounding boxes, and masks
logits = outputs.logits # use the `post_process_panoptic` method of `DetrFeatureExtractor` to convert to COCO format
bboxes = outputs.pred_boxes processed_sizes = torch.as_tensor(inputs["pixel_values"].shape[-2:]).unsqueeze(0)
masks = outputs.pred_masks result = feature_extractor.post_process_panoptic(outputs, processed_sizes)[0]
# the segmentation is stored in a special-format png
panoptic_seg = Image.open(io.BytesIO(result["png_string"]))
panoptic_seg = numpy.array(panoptic_seg, dtype=numpy.uint8)
# retrieve the ids corresponding to each mask
panoptic_seg_id = rgb_to_id(panoptic_seg)
``` ```
Currently, both the feature extractor and model support PyTorch. Currently, both the feature extractor and model support PyTorch.