From ce29d187bc8d84633d7c589ccfce4a3503e32efe Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Aitor=20Garc=C3=ADa=20Rey?= Date: Fri, 14 Apr 2023 13:35:32 +0200 Subject: [PATCH 1/4] feat(docs): Include documentation about how to boot the app locally. --- README.md | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index e65881b..61cabf8 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ Prompt engineering is a relatively new discipline for developing and optimizing prompts to efficiently use language models (LMs) for a wide variety of applications and research topics. Prompt engineering skills help to better understand the capabilities and limitations of large language models (LLMs). Researchers use prompt engineering to improve the capacity of LLMs on a wide range of common and complex tasks such as question answering and arithmetic reasoning. Developers use prompt engineering to design robust and effective prompting techniques that interface with LLMs and other tools. -Motivated by the high interest in developing with LLMs, we have created this new prompt engineering guide that contains all the latest papers, learning guides, lectures, references, and tools related to prompt engineering. +Motivated by the high interest in developing with LLMs, we have created this new prompt engineering guide that contains all the latest papers, learning guides, lectures, references, and tools related to prompt engineering. Happy Prompting! @@ -29,10 +29,21 @@ We have published a 1 hour lecture that provides a comprehensive overview of pro - [Notebook with code](https://github.com/dair-ai/Prompt-Engineering-Guide/blob/main/notebooks/pe-lecture.ipynb) - [Slides](https://github.com/dair-ai/Prompt-Engineering-Guide/blob/main/lecture/Prompt-Engineering-Lecture-Elvis.pdf) +--- +## Running the guide locally + +To run the guide locally, for example to check the correct implementation of a new translation, you will need to: + +1. Install Node >=18.0.0 +1. Install `pnpm` if not present in your system. Check [here](https://pnpm.io/installation) for detailed instructions. +1. Install the dependencies: `pnpm i next react react-dom nextra nextra-theme-docs` +1. Boot the guide with `pnpm dev` +2. Browse the guide at `http://localhost:3000/` + --- ## Guides -Please note that the guides below are now outdated. You can find the most up-to-date guides on our new website [https://www.promptingguide.ai/](https://www.promptingguide.ai/). +Please note that the guides below are now outdated. You can find the most up-to-date guides on our new website [https://www.promptingguide.ai/](https://www.promptingguide.ai/). - [Prompt Engineering - Introduction](/guides/prompts-intro.md) - [Prompt Engineering - Basic Prompting](/guides/prompts-basic-usage.md) @@ -44,7 +55,7 @@ Please note that the guides below are now outdated. You can find the most up-to- - [Prompt Engineering - Miscellaneous Topics](/guides/prompts-miscellaneous.md) - [Prompt Engineering - Papers](/pages/papers.en.mdx) - [Prompt Engineering - Tools](/pages/tools.en.mdx) -- [Prompt Engineering - Datasets](/pages/datasets.en.mdx) +- [Prompt Engineering - Datasets](/pages/datasets.en.mdx) - [Prompt Engineering - Additional Readings](/pages/readings.en.mdx) --- From 6a5669a9e2a151040e663bcf286d63074ae4949a Mon Sep 17 00:00:00 2001 From: Tao Li Date: Fri, 14 Apr 2023 22:33:30 -0700 Subject: [PATCH 2/4] Update introduction.en.mdx You do not need to spell out LLMs twice. --- pages/introduction.en.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pages/introduction.en.mdx b/pages/introduction.en.mdx index db33b02..3724f0f 100644 --- a/pages/introduction.en.mdx +++ b/pages/introduction.en.mdx @@ -2,6 +2,6 @@ Prompt engineering is a relatively new discipline for developing and optimizing prompts to efficiently use language models (LMs) for a wide variety of applications and research topics. Prompt engineering skills help to better understand the capabilities and limitations of large language models (LLMs). Researchers use prompt engineering to improve the capacity of LLMs on a wide range of common and complex tasks such as question answering and arithmetic reasoning. Developers use prompt engineering to design robust and effective prompting techniques that interface with LLMs and other tools. -This guide covers the basics of prompts to provide a rough idea of how to use prompts to interact and instruct large language models (LLMs). +This guide covers the basics of prompts to provide a rough idea of how to use prompts to interact and instruct LLMs. -All examples are tested with `text-davinci-003` (using OpenAI's playground) unless otherwise specified. It uses the default configurations, i.e., `temperature=0.7` and `top-p=1`. \ No newline at end of file +All examples are tested with `text-davinci-003` (using OpenAI's playground) unless otherwise specified. It uses the default configurations, i.e., `temperature=0.7` and `top-p=1`. From 71d29386e8d02d7ee2a01c9211b310791a4ea973 Mon Sep 17 00:00:00 2001 From: Tao Li Date: Fri, 14 Apr 2023 23:09:48 -0700 Subject: [PATCH 3/4] Update settings.en.mdx Avoid future tense. Avoid using first person (like we). Instead use second person directly to address the reader. --- pages/introduction/settings.en.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pages/introduction/settings.en.mdx b/pages/introduction/settings.en.mdx index b84e42f..408987a 100644 --- a/pages/introduction/settings.en.mdx +++ b/pages/introduction/settings.en.mdx @@ -1,11 +1,11 @@ # LLM Settings -When working with prompts, you will be interacting with the LLM via an API or directly. You can configure a few parameters to get different results for your prompts. +When working with prompts, you interact with the LLM via an API or directly. You can configure a few parameters to get different results for your prompts. -**Temperature** - In short, the lower the `temperature` the more deterministic the results in the sense that the highest probable next token is always picked. Increasing temperature could lead to more randomness encouraging more diverse or creative outputs. We are essentially increasing the weights of the other possible tokens. In terms of application, we might want to use a lower temperature value for tasks like fact-based QA to encourage more factual and concise responses. For poem generation or other creative tasks, it might be beneficial to increase the temperature value. +**Temperature** - In short, the lower the `temperature`, the more deterministic the results in the sense that the highest probable next token is always picked. Increasing temperature could lead to more randomness, which encourages more diverse or creative outputs. You are essentially increasing the weights of the other possible tokens. In terms of application, you might want to use a lower temperature value for tasks like fact-based QA to encourage more factual and concise responses. For poem generation or other creative tasks, it might be beneficial to increase the temperature value. **Top_p** - Similarly, with `top_p`, a sampling technique with temperature called nucleus sampling, you can control how deterministic the model is at generating a response. If you are looking for exact and factual answers keep this low. If you are looking for more diverse responses, increase to a higher value. The general recommendation is to alter one, not both. -Before starting with some basic examples, keep in mind that your results may vary depending on the version of LLM you are using. \ No newline at end of file +Before starting with some basic examples, keep in mind that your results may vary depending on the version of LLM you use. From 90da7356a8a7154be96532100ef0a493188d14e8 Mon Sep 17 00:00:00 2001 From: Elvis Saravia Date: Sun, 16 Apr 2023 16:19:42 -0600 Subject: [PATCH 4/4] added new papers --- pages/models/chatgpt.en.mdx | 9 +++++++++ pages/models/gpt-4.en.mdx | 3 +++ pages/papers.en.mdx | 6 ++++++ pages/readings.en.mdx | 1 + pages/tools.en.mdx | 1 + 5 files changed, 20 insertions(+) diff --git a/pages/models/chatgpt.en.mdx b/pages/models/chatgpt.en.mdx index 7ec5531..760653c 100644 --- a/pages/models/chatgpt.en.mdx +++ b/pages/models/chatgpt.en.mdx @@ -145,6 +145,15 @@ The current recommendation for `gpt-3.5-turbo-0301` is to add instructions in th --- ## References +- [ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learning](https://arxiv.org/abs/2304.06588) (April 2023) +- [ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning](https://arxiv.org/abs/2304.05613) (April 2023) +- [Distinguishing ChatGPT(-3.5, -4)-generated and human-written papers through Japanese stylometric analysis](https://arxiv.org/abs/2304.05534) (April 2023) +- [Zero-shot Temporal Relation Extraction with ChatGPT](https://arxiv.org/abs/2304.05454) (April 2023) +- [Can ChatGPT and Bard Generate Aligned Assessment Items? A Reliability Analysis against Human Performance](https://arxiv.org/abs/2304.05372) (April 2023) +- [Are Large Language Models Ready for Healthcare? A Comparative Study on Clinical Language Understanding](https://arxiv.org/abs/2304.05368) (April 2023) +- [The Wall Street Neophyte: A Zero-Shot Analysis of ChatGPT Over MultiModal Stock Movement Prediction Challenges](https://arxiv.org/abs/2304.05351) (April 2023) +- [Toxicity in ChatGPT: Analyzing Persona-assigned Language Models](https://arxiv.org/abs/2304.05335) (April 2023) +- [Multi-step Jailbreaking Privacy Attacks on ChatGPT](https://arxiv.org/abs/2304.05197) (April 2023) - [Is ChatGPT a Good Sentiment Analyzer? A Preliminary Study](https://arxiv.org/abs/2304.04339) (April 2023) - [A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understanding](https://arxiv.org/abs/2304.04256) (April 2023) - [Extractive Summarization via ChatGPT for Faithful Summary Generation](https://arxiv.org/abs/2304.04193) (April 2023) diff --git a/pages/models/gpt-4.en.mdx b/pages/models/gpt-4.en.mdx index 4199f7b..11bffa0 100644 --- a/pages/models/gpt-4.en.mdx +++ b/pages/models/gpt-4.en.mdx @@ -160,6 +160,9 @@ Coming soon! ## References / Papers +- [chatIPCC: Grounding Conversational AI in Climate Science](https://arxiv.org/abs/2304.05510) (April 2023) +- [Galactic ChitChat: Using Large Language Models to Converse with Astronomy Literature](https://arxiv.org/abs/2304.05406) (April 2023) +- [Emergent autonomous scientific research capabilities of large language models](https://arxiv.org/abs/2304.05332) (April 2023) - [Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4](https://arxiv.org/abs/2304.03439) (April 2023) - [Instruction Tuning with GPT-4](https://arxiv.org/abs/2304.03277) (April 2023) - [Evaluating GPT-4 and ChatGPT on Japanese Medical Licensing Examinations](https://arxiv.org/abs/2303.18027) (April 2023) diff --git a/pages/papers.en.mdx b/pages/papers.en.mdx index 0fbf640..2c7ab81 100644 --- a/pages/papers.en.mdx +++ b/pages/papers.en.mdx @@ -4,6 +4,7 @@ The following are the latest papers (sorted by release date) on prompt engineeri ## Overviews + - [One Small Step for Generative AI, One Giant Leap for AGI: A Complete Survey on ChatGPT in AIGC Era](https://arxiv.org/abs/2304.06488) (April 2023) - [A Bibliometric Review of Large Language Models Research from 2017 to 2023](https://arxiv.org/abs/2304.02020) (April 2023) - [A Survey of Large Language Models](https://arxiv.org/abs/2303.18223) (April 2023) - [Nature Language Reasoning, A Survey](https://arxiv.org/abs/2303.14725) (Mar 2023) @@ -17,6 +18,8 @@ The following are the latest papers (sorted by release date) on prompt engineeri ## Approaches + - [Boosted Prompt Ensembles for Large Language Models](https://arxiv.org/abs/2304.05970) (April 2023) + - [Global Prompt Cell: A Portable Control Module for Effective Prompt](https://arxiv.org/abs/2304.05642) (April 2023) - [Why think step-by-step? Reasoning emerges from the locality of experience](https://arxiv.org/abs/2304.03843) (April 2023) - [Revisiting Automated Prompting: Are We Actually Doing Better?](https://arxiv.org/abs/2304.03609) (April 2023) - [REFINER: Reasoning Feedback on Intermediate Representations](https://arxiv.org/abs/2304.01904) (April 2023) @@ -126,6 +129,9 @@ The following are the latest papers (sorted by release date) on prompt engineeri ## Applications + - [Are LLMs All You Need for Task-Oriented Dialogue?](https://arxiv.org/abs/2304.06556) (April 2023) + - [HiPrompt: Few-Shot Biomedical Knowledge Fusion via Hierarchy-Oriented Prompting](https://arxiv.org/abs/2304.05973) (April 2023) + - [Approximating Human Evaluation of Social Chatbots with Prompting](https://arxiv.org/abs/2304.05253) (April 2023) - [Automated Reading Passage Generation with OpenAI's Large Language Model](https://arxiv.org/abs/2304.04616) (April 2023) - [WebBrain: Learning to Generate Factually Correct Articles for Queries by Grounding on Large Web Corpus](https://arxiv.org/abs/2304.04358) (April 2023) - [Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition](https://arxiv.org/abs/2304.04704) (April 2023) diff --git a/pages/readings.en.mdx b/pages/readings.en.mdx index 4863b43..f6cd2ea 100644 --- a/pages/readings.en.mdx +++ b/pages/readings.en.mdx @@ -62,6 +62,7 @@ - [Language Model Behavior: A Comprehensive Survey](https://arxiv.org/abs/2303.11504) - [Learn Prompting](https://learnprompting.org) - [Learning Prompt](https://github.com/thinkingjimmy/Learning-Prompt) +- [LINGO : Visually Debiasing Natural Language Instructions to Support Task Diversity](https://arxiv.org/abs/2304.06184) - [Meet Claude: Anthropic’s Rival to ChatGPT](https://scale.com/blog/chatgpt-vs-claude) - [Methods of prompt programming](https://generative.ink/posts/methods-of-prompt-programming) - [Mysteries of mode collapse](https://www.lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-of-mode-collapse) diff --git a/pages/tools.en.mdx b/pages/tools.en.mdx index bc7b0ca..24d0f02 100644 --- a/pages/tools.en.mdx +++ b/pages/tools.en.mdx @@ -10,6 +10,7 @@ - [Dyno](https://trydyno.com) - [EmergentMind](https://www.emergentmind.com) - [EveryPrompt](https://www.everyprompt.com) +- [Guardrails](https://github.com/ShreyaR/guardrails) - [GPT Index](https://github.com/jerryjliu/gpt_index) - [GPTTools](https://gpttools.com/comparisontool) - [hwchase17/adversarial-prompts](https://github.com/hwchase17/adversarial-prompts)