From 96afae46dc943c454eb9277e808def0c7c81c6e8 Mon Sep 17 00:00:00 2001 From: Niels Rogge Date: Thu, 5 May 2022 07:01:31 +0000 Subject: [PATCH] Update README.md --- README.md | 127 +++++++++++++++++++++++++++--------------------------- 1 file changed, 64 insertions(+), 63 deletions(-) diff --git a/README.md b/README.md index e8fb2f1..0d3a346 100644 --- a/README.md +++ b/README.md @@ -1,64 +1,65 @@ ---- -language: en -tags: -- tapex -license: mit ---- - -# TAPEX (base-sized model) - -TAPEX was proposed in [TAPEX: Table Pre-training via Learning a Neural SQL Executor](https://arxiv.org/abs/2107.07653) by Qian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi, Zeqi Lin, Weizhu Chen, Jian-Guang Lou. The original repo can be found [here](https://github.com/microsoft/Table-Pretraining). - -## Model description - -TAPEX (**Ta**ble **P**re-training via **Ex**ecution) is a conceptually simple and empirically powerful pre-training approach to empower existing models with *table reasoning* skills. TAPEX realizes table pre-training by learning a neural SQL executor over a synthetic corpus, which is obtained by automatically synthesizing executable SQL queries. - -TAPEX is based on the BART architecture, the transformer encoder-encoder (seq2seq) model with a bidirectional (BERT-like) encoder and an autoregressive (GPT-like) decoder. - -## Intended Uses - -You can use the raw model for simulating neural SQL execution, i.e., employ TAPEX to execute a SQL query on a given table. However, the model is mostly meant to be fine-tuned on a supervised dataset. Currently TAPEX can be fine-tuned to tackle table question answering tasks and table fact verification tasks. See the [model hub](https://huggingface.co/models?search=tapex) to look for fine-tuned versions on a task that interests you. - -### How to Use - -Here is how to use this model in transformers: - -```python -from transformers import TapexTokenizer, BartForConditionalGeneration -import pandas as pd - -tokenizer = TapexTokenizer.from_pretrained("microsoft/tapex-base") -model = BartForConditionalGeneration.from_pretrained("microsoft/tapex-base") - -data = { - "year": [1896, 1900, 1904, 2004, 2008, 2012], - "city": ["athens", "paris", "st. louis", "athens", "beijing", "london"] -} -table = pd.DataFrame.from_dict(data) - -# tapex accepts uncased input since it is pre-trained on the uncased corpus -query = "select year where city = beijing" -encoding = tokenizer(table=table, query=query, return_tensors="pt") - -outputs = model.generate(**encoding) - -print(tokenizer.batch_decode(outputs, skip_special_tokens=True)) -# ['2008'] -``` - -### How to Fine-tuning - -Please find the fine-tuning script [here](https://github.com/SivilTaram/transformers/tree/add_tapex_bis/examples/research_projects/tapex). - -### BibTeX entry and citation info - -```bibtex -@inproceedings{ - liu2022tapex, - title={{TAPEX}: Table Pre-training via Learning a Neural {SQL} Executor}, - author={Qian Liu and Bei Chen and Jiaqi Guo and Morteza Ziyadi and Zeqi Lin and Weizhu Chen and Jian-Guang Lou}, - booktitle={International Conference on Learning Representations}, - year={2022}, - url={https://openreview.net/forum?id=O50443AsCP} -} +--- +language: en +tags: +- tapex +- table-question-answering +license: mit +--- + +# TAPEX (base-sized model) + +TAPEX was proposed in [TAPEX: Table Pre-training via Learning a Neural SQL Executor](https://arxiv.org/abs/2107.07653) by Qian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi, Zeqi Lin, Weizhu Chen, Jian-Guang Lou. The original repo can be found [here](https://github.com/microsoft/Table-Pretraining). + +## Model description + +TAPEX (**Ta**ble **P**re-training via **Ex**ecution) is a conceptually simple and empirically powerful pre-training approach to empower existing models with *table reasoning* skills. TAPEX realizes table pre-training by learning a neural SQL executor over a synthetic corpus, which is obtained by automatically synthesizing executable SQL queries. + +TAPEX is based on the BART architecture, the transformer encoder-encoder (seq2seq) model with a bidirectional (BERT-like) encoder and an autoregressive (GPT-like) decoder. + +## Intended Uses + +You can use the raw model for simulating neural SQL execution, i.e., employ TAPEX to execute a SQL query on a given table. However, the model is mostly meant to be fine-tuned on a supervised dataset. Currently TAPEX can be fine-tuned to tackle table question answering tasks and table fact verification tasks. See the [model hub](https://huggingface.co/models?search=tapex) to look for fine-tuned versions on a task that interests you. + +### How to Use + +Here is how to use this model in transformers: + +```python +from transformers import TapexTokenizer, BartForConditionalGeneration +import pandas as pd + +tokenizer = TapexTokenizer.from_pretrained("microsoft/tapex-base") +model = BartForConditionalGeneration.from_pretrained("microsoft/tapex-base") + +data = { + "year": [1896, 1900, 1904, 2004, 2008, 2012], + "city": ["athens", "paris", "st. louis", "athens", "beijing", "london"] +} +table = pd.DataFrame.from_dict(data) + +# tapex accepts uncased input since it is pre-trained on the uncased corpus +query = "select year where city = beijing" +encoding = tokenizer(table=table, query=query, return_tensors="pt") + +outputs = model.generate(**encoding) + +print(tokenizer.batch_decode(outputs, skip_special_tokens=True)) +# ['2008'] +``` + +### How to Fine-tuning + +Please find the fine-tuning script [here](https://github.com/SivilTaram/transformers/tree/add_tapex_bis/examples/research_projects/tapex). + +### BibTeX entry and citation info + +```bibtex +@inproceedings{ + liu2022tapex, + title={{TAPEX}: Table Pre-training via Learning a Neural {SQL} Executor}, + author={Qian Liu and Bei Chen and Jiaqi Guo and Morteza Ziyadi and Zeqi Lin and Weizhu Chen and Jian-Guang Lou}, + booktitle={International Conference on Learning Representations}, + year={2022}, + url={https://openreview.net/forum?id=O50443AsCP} +} ``` \ No newline at end of file