From d340591382198ec7528c8c3474dd3e77f44f9eed Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Rub=C3=A8n=20Tito?= Date: Fri, 24 Feb 2023 08:01:32 +0000 Subject: [PATCH] Update README.md --- README.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/README.md b/README.md index 9a1eaa7..4ba7139 100644 --- a/README.md +++ b/README.md @@ -44,6 +44,17 @@ end_idx = torch.argmax(outputs.end_logits, axis=1) answers = self.processor.tokenizer.decode(input_tokens[start_idx: end_idx+1]).strip() ``` +## Metrics +**Average Normalized Levenshtein Similarity (ANLS)** + +The standard metric for text-based VQA tasks (ST-VQA and DocVQA). It evaluates the method's reasoning capabilities while smoothly penalizes OCR recognition errors. +Check [Scene Text Visual Question Answering](https://arxiv.org/abs/1905.13648) for detailed information. + +**Answer Page Prediction Accuracy (APPA)** + +In the MP-DocVQA task, the models can provide the index of the page where the information required to answer the question is located. For this subtask accuracy is used to evaluate the predictions: i.e. if the predicted page is correct or not. +Check [Hierarchical multimodal transformers for Multi-Page DocVQA](https://arxiv.org/abs/2212.05935) for detailed information. + ## Model results Extended experimentation can be found in Table 2 of [Hierarchical multimodal transformers for Multi-Page DocVQA](https://arxiv.org/pdf/2212.05935.pdf).