From 5affa20ba4270832b95f52c39c640a02ad19f31d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=B0=8F=E8=87=A3=E5=AD=90=E5=90=83=E5=A4=A7=E6=A9=99?= =?UTF-8?q?=E5=AD=90?= Date: Fri, 3 Feb 2023 17:33:41 +0800 Subject: [PATCH 1/2] sort the websites and data sets by name, fix several typos, and add a few entries. --- README.md | 174 +++++++++++++++++++++++++++++------------------------- 1 file changed, 92 insertions(+), 82 deletions(-) diff --git a/README.md b/README.md index ee34c93..8de12c8 100644 --- a/README.md +++ b/README.md @@ -14,17 +14,19 @@ This guide contains a set of papers, learning guides, and tools related to promp - [Blog, Guides, Tutorials and Other Readings](#blog-guides-tutorials-and-other-readings) ## Papers +#### (Sorted by Release Date) - Surveys / Overviews: + - [A Survey for In-context Learning](https://arxiv.org/abs/2301.00234) (Dec 2022) - [Emergent Abilities of Large Language Models](https://arxiv.org/abs/2206.07682) (Jun 2022) - [A Taxonomy of Prompt Modifiers for Text-To-Image Generation](https://arxiv.org/abs/2204.13988) (Apr 2022) - [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://arxiv.org/abs/2107.13586) (Jul 2021) - - Approaches/Techniques: - - [Multimodal Chain-of-Thought Reasoning in Language Models](https://arxiv.org/abs/2302.00923) (Feb 2022) - - [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://arxiv.org/abs/2302.00093) (Feb 2022) - - [Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models](https://arxiv.org/abs/2302.00618) (Feb 2022) + + - [Multimodal Chain-of-Thought Reasoning in Language Models](https://arxiv.org/abs/2302.00923) (Feb 2023) + - [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://arxiv.org/abs/2302.00093) (Feb 2023) + - [Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models](https://arxiv.org/abs/2302.00618) (Feb 2023) - [Progressive Prompts: Continual Learning for Language Models](https://arxiv.org/abs/2301.12314) (Jan 2023) - [Batch Prompting: Efficient Inference with LLM APIs](https://arxiv.org/abs/2301.08721) (Jan 2023) - [On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning](https://arxiv.org/abs/2212.08061) (Dec 2022) @@ -68,117 +70,125 @@ This guide contains a set of papers, learning guides, and tools related to promp - [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://arxiv.org/abs/2010.15980) (Oct 2020) - [Language Models are Few-Shot Learners](https://arxiv.org/abs/2005.14165) (May 2020) - [How Can We Know What Language Models Know?](https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00324/96460/How-Can-We-Know-What-Language-Models-Know) (July 2020) - - Applications: + - [Legal Prompt Engineering for Multilingual Legal Judgement Prediction](https://arxiv.org/abs/2212.02199) (Dec 2022) - [Investigating Prompt Engineering in Diffusion Models](https://arxiv.org/abs/2211.15462) (Nov 2022) - [Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language](https://arxiv.org/abs/2210.15157) (Oct 2022) - [Piloting Copilot and Codex: Hot Temperature, Cold Prompts, or Black Magic?](https://arxiv.org/abs/2210.14699) (Oct 2022) - [Plot Writing From Scratch Pre-Trained Language Models](https://aclanthology.org/2022.inlg-main.5/) (July 2022) - - Collections: + + - [Chain-of-ThoughtsPapers](https://github.com/Timothyxxx/Chain-of-ThoughtsPapers) - [Papers with Code](https://paperswithcode.com/task/prompt-engineering) - [Prompt Papers](https://github.com/thunlp/PromptPapers#papers) - ## Tools & Libraries +#### (Sorted by Name) -- [OpenAI Playground](https://beta.openai.com/playground) -- [GPTTools](https://gpttools.com/comparisontool) -- [LangChain](https://github.com/hwchase17/langchain) -- [ThoughtSource](https://github.com/OpenBioLink/ThoughtSource) -- [EveryPrompt](https://www.everyprompt.com/) -- [DUST](https://dust.tt/) -- [Dyno](https://trydyno.com/) -- [Metaprompt](https://metaprompt.vercel.app/?task=gpt) -- [Prompts.ai](https://github.com/sevazhidkov/prompts-ai) -- [Lexica](https://lexica.art/) -- [Scale SpellBook](https://scale.com/spellbook) -- [Interactive Composition Explorer](https://github.com/oughtinc/ice) -- [LearnGPT](https://www.learngpt.com/) -- [hwchase17/adversarial-prompts](https://github.com/hwchase17/adversarial-prompts) -- [Promptable](https://promptable.ai/) -- [GPT Index](https://github.com/jerryjliu/gpt_index) -- [Prompt Base](https://promptbase.com/) -- [Playground](https://playgroundai.com/) -- [OpenPrompt](https://github.com/thunlp/OpenPrompt) -- [Visual Prompt Builder](https://tools.saxifrage.xyz/prompt) -- [Prompt Generator for OpenAI's DALL-E 2](http://dalle2-prompt-generator.s3-website-us-west-2.amazonaws.com/) - [AI Test Kitchen](https://aitestkitchen.withgoogle.com/) - [betterprompt](https://github.com/krrishdholakia/betterprompt) -- [Prompt Engine](https://github.com/microsoft/prompt-engine) -- [PromptSource](https://github.com/bigscience-workshop/promptsource) -- [sharegpt](https://sharegpt.com/) - [DreamStudio](https://beta.dreamstudio.ai/) +- [DUST](https://dust.tt/) +- [Dyno](https://trydyno.com/) +- [EveryPrompt](https://www.everyprompt.com/) +- [GPT Index](https://github.com/jerryjliu/gpt_index) +- [GPTTools](https://gpttools.com/comparisontool) +- [hwchase17/adversarial-prompts](https://github.com/hwchase17/adversarial-prompts) +- [Interactive Composition Explorer](https://github.com/oughtinc/ice) +- [LangChain](https://github.com/hwchase17/langchain) +- [LearnGPT](https://www.learngpt.com/) +- [Lexica](https://lexica.art/) +- [Metaprompt](https://metaprompt.vercel.app/?task=gpt) +- [OpenAI Playground](https://beta.openai.com/playground) +- [OpenPrompt](https://github.com/thunlp/OpenPrompt) +- [Playground](https://playgroundai.com/) +- [Prompt Base](https://promptbase.com/) +- [Prompt Engine](https://github.com/microsoft/prompt-engine) +- [Prompt Generator for OpenAI's DALL-E 2](http://dalle2-prompt-generator.s3-website-us-west-2.amazonaws.com/) +- [Promptable](https://promptable.ai/) - [PromptInject](https://github.com/agencyenterprise/PromptInject) +- [Prompts.ai](https://github.com/sevazhidkov/prompts-ai) +- [PromptSource](https://github.com/bigscience-workshop/promptsource) +- [Scale SpellBook](https://scale.com/spellbook) +- [sharegpt](https://sharegpt.com/) +- [ThoughtSource](https://github.com/OpenBioLink/ThoughtSource) +- [Visual Prompt Builder](https://tools.saxifrage.xyz/prompt) ## Datasets +#### (Sorted by Name) + +- [Anthropic's Red Team dataset](https://github.com/anthropics/hh-rlhf/tree/master/red-team-attempts), [(paper)](https://arxiv.org/abs/2209.07858) +- [Awesome ChatGPT Prompts](https://huggingface.co/datasets/fka/awesome-chatgpt-prompts) +- [DiffusionDB](https://github.com/poloclub/diffusiondb) +- [Midjourney Prompts](https://huggingface.co/datasets/succinctly/midjourney-prompts) +- [P3 - Public Pool of Prompts](https://huggingface.co/datasets/bigscience/P3) - [PartiPrompts](https://parti.research.google/) - [Real Toxicity Prompts](https://allenai.org/data/real-toxicity-prompts) -- [DiffusionDB](https://github.com/poloclub/diffusiondb) -- [P3 - Public Pool of Prompts](https://huggingface.co/datasets/bigscience/P3) -- [WritingPrompts](WritingPrompts) -- [Midjourney Prompts](https://huggingface.co/datasets/succinctly/midjourney-prompts) -- [Awesome ChatGPT Prompts](https://huggingface.co/datasets/fka/awesome-chatgpt-prompts) - [Stable Diffusion Dataset](https://huggingface.co/datasets/Gustavosta/Stable-Diffusion-Prompts) -- [Anthropic's Red Team dataset](https://github.com/anthropics/hh-rlhf/tree/master/red-team-attempts), [(paper)](https://arxiv.org/abs/2209.07858) +- [WritingPrompts](https://www.reddit.com/r/WritingPrompts/) ## Blog, Guides, Tutorials and Other Readings -- [Prompt injection to read out the secret OpenAI API key](https://twitter.com/ludwig_stumpp/status/1619701277419794435?s=20&t=GtoMlmYCSt-UmvjqJVbBSA) -- [The ChatGPT Prompt Book](https://docs.google.com/presentation/d/17b_ocq-GL5lhV_bYSShzUgxL02mtWDoiw9xEroJ5m3Q/edit#slide=id.gc6f83aa91_0_79) +#### (Sorted by Name) + +- [3 Principles for prompt engineering with GPT-3](https://www.linkedin.com/pulse/3-principles-prompt-engineering-gpt-3-ben-whately/) +- [A beginner-friendly guide to generative language models - LaMBDA guide](https://aitestkitchen.withgoogle.com/how-lamda-works) +- [A Complete Introduction to Prompt Engineering for Large Language Models](https://www.mihaileric.com/posts/a-complete-introduction-to-prompt-engineering/) +- [A Generic Framework for ChatGPT Prompt Engineering](https://medium.com/@thorbjoern.heise/a-generic-framework-for-chatgpt-prompt-engineering-7097f6513a0b) +- [AI Content Generation](https://www.jonstokes.com/p/ai-content-generation-part-1-machine) +- [Awesome ChatGPT Prompts](https://github.com/f/awesome-chatgpt-prompts) +- [Best 100+ Stable Diffusion Prompts](https://mpost.io/best-100-stable-diffusion-prompts-the-most-beautiful-ai-text-to-image-prompts/) +- [Best practices for prompt engineering with OpenAI API](https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api) +- [CMU Advanced NLP 2022: Prompting](https://youtube.com/watch?v=5ef83Wljm-M&feature=shares) +- [Curtis64's set of prompt gists](https://gist.github.com/Curtis-64) +- [DALL·E 2 Prompt Engineering Guide](https://docs.google.com/document/d/11WlzjBT0xRpQhP9tFMtxzd0q6ANIdHPUBkMV-YB043U/edit#) +- [DALLE Prompt Book](https://dallery.gallery/the-dalle-2-prompt-book/) +- [Exploiting GPT-3 Prompts](https://twitter.com/goodside/status/1569128808308957185) +- [Exploring Prompt Injection Attacks](Exploring Prompt Injection Attacks) +- [Extrapolating to Unnatural Language Processing with GPT-3's In-context Learning: The Good, the Bad, and the Mysterious](http://ai.stanford.edu/blog/in-context-learning/) +- [Generative AI with Cohere: Part 1 - Model Prompting](https://txt.cohere.ai/generative-ai-part-1/) +- [Giving GPT-3 a Turing Test](https://lacker.io/ai/2020/07/06/giving-gpt-3-a-turing-test.html) +- [GPT3 and Prompts: A quick primer](https://buildspace.so/notes/intro-to-gpt3-prompts) +- [How to Draw Anything](https://andys.page/posts/how-to-draw/) +- [How to get images that don't suck](https://www.reddit.com/r/StableDiffusion/comments/x41n87/how_to_get_images_that_dont_suck_a/) +- [How to write good prompts](https://andymatuschak.org/prompts/) +- [Language Models and Prompt Engineering: Systematic Survey of Prompting Methods in NLP](https://youtube.com/watch?v=OsbUfL8w-mo&feature=shares) +- [Learn Prompting](https://learnprompting.org/) +- [Methods of prompt programming](https://generative.ink/posts/methods-of-prompt-programming/) +- [Mysteries of mode collapse](https://www.lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-of-mode-collapse) +- [NLP for Text-to-Image Generators: Prompt Analysis](https://heartbeat.comet.ml/nlp-for-text-to-image-generators-prompt-analysis-part-1-5076a44d8365) +- [Notes for Prompt Engineering by sw-yx](https://github.com/sw-yx/ai-notes) - [Pretrain, Prompt, Predict - A New Paradigm for NLP](http://pretrain.nlpedia.ai/) - [Prompt Engineering 101 - Introduction and resources](https://www.linkedin.com/pulse/prompt-engineering-101-introduction-resources-amatriain/) -- [Prompt Engineering by co:here](https://docs.cohere.ai/docs/prompt-engineering) -- [Generative AI with Cohere: Part 1 - Model Prompting](https://txt.cohere.ai/generative-ai-part-1/) -- [Prompt Engineering by Microsoft](https://microsoft.github.io/prompt-engineering/) -- [Best practices for prompt engineering with OpenAI API](https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api) -- [Exploring Prompt Injection Attacks](Exploring Prompt Injection Attacks) -- [Start with an Instruction](https://beta.openai.com/docs/quickstart/start-with-an-instruction) -- [CMU Advanced NLP 2022: Prompting](https://youtube.com/watch?v=5ef83Wljm-M&feature=shares) - [Prompt Engineering 101: Autocomplete, Zero-shot, One-shot, and Few-shot prompting](https://youtube.com/watch?v=v2gD8BHOaX4&feature=shares) -- [Prompt engineering davinci-003 on our own docs for automated support (Part I)](https://www.patterns.app/blog/2022/12/21/finetune-llm-tech-support/) -- [DALLE Prompt Book](https://dallery.gallery/the-dalle-2-prompt-book/) -- [DALL·E 2 Prompt Engineering Guide](https://docs.google.com/document/d/11WlzjBT0xRpQhP9tFMtxzd0q6ANIdHPUBkMV-YB043U/edit#) -- [Prompt injection attacks against GPT-3](https://simonwillison.net/2022/Sep/12/prompt-injection/) -- [Reverse Prompt Engineering for Fun and (no) Profit](https://lspace.swyx.io/p/reverse-prompt-eng) -- [Language Models and Prompt Engineering: Systematic Survey of Prompting Methods in NLP](https://youtube.com/watch?v=OsbUfL8w-mo&feature=shares) -- [A Complete Introduction to Prompt Engineering for Large Language Models](https://www.mihaileric.com/posts/a-complete-introduction-to-prompt-engineering/) -- [Learn Prompting](https://learnprompting.org/) -- [3 Principles for prompt engineering with GPT-3](https://www.linkedin.com/pulse/3-principles-prompt-engineering-gpt-3-ben-whately/) -- [Extrapolating to Unnatural Language Processing with GPT-3's In-context Learning: The Good, the Bad, and the Mysterious](http://ai.stanford.edu/blog/in-context-learning/) -- [Prompt Engineering Topic by GitHub](https://github.com/topics/prompt-engineering) -- [Prompt Engineering Template](https://docs.google.com/spreadsheets/d/1-snKDn38-KypoYCk9XLPg799bHcNFSBAVu2HVvFEAkA/edit#gid=0) -- [Awesome ChatGPT Prompts](https://github.com/f/awesome-chatgpt-prompts) -- [Prompt Engineering: From Words to Art](https://www.saxifrage.xyz/post/prompt-engineering) -- [NLP for Text-to-Image Generators: Prompt Analysis](https://heartbeat.comet.ml/nlp-for-text-to-image-generators-prompt-analysis-part-1-5076a44d8365) -- [Mysteries of mode collapse](https://www.lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-of-mode-collapse) -- [GPT3 and Prompts: A quick primer](https://buildspace.so/notes/intro-to-gpt3-prompts) -- [Prompt Engineering in GPT-3](https://www.analyticsvidhya.com/blog/2022/05/prompt-engineering-in-gpt-3/) -- [Talking to machines: prompt engineering & injection](https://artifact-research.com/artificial-intelligence/talking-to-machines-prompt-engineering-injection/) -- [A beginner-friendly guide to generative language models - LaMBDA guide](https://aitestkitchen.withgoogle.com/how-lamda-works) -- [Giving GPT-3 a Turing Test](https://lacker.io/ai/2020/07/06/giving-gpt-3-a-turing-test.html) -- [Prompts as Programming by Gwern](https://www.gwern.net/GPT-3#prompts-as-programming) -- [AI Content Generation](https://www.jonstokes.com/p/ai-content-generation-part-1-machine) -- [How to Draw Anything](https://andys.page/posts/how-to-draw/) -- [How to write good prompts](https://andymatuschak.org/prompts/) -- [Exploiting GPT-3 Prompts](https://twitter.com/goodside/status/1569128808308957185) -- [Prompting Methods with Language Models and Their Applications to Weak Supervision](https://snorkel.ai/prompting-methods-with-language-models-nlp/) -- [Simulators](https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators) -- [How to get images that don't suck](https://www.reddit.com/r/StableDiffusion/comments/x41n87/how_to_get_images_that_dont_suck_a/) -- [Best 100+ Stable Diffusion Prompts](https://mpost.io/best-100-stable-diffusion-prompts-the-most-beautiful-ai-text-to-image-prompts/) -- [Using GPT-Eliezer against ChatGPT Jailbreaking](https://www.alignmentforum.org/posts/pNcFYZnPdXyL2RfgA/using-gpt-eliezer-against-chatgpt-jailbreaking) -- [Notes for Prompt Engineering by sw-yx](https://github.com/sw-yx/ai-notes) -- [Prompt Engineering Guide: How to Engineer the Perfect Prompts](https://richardbatt.co.uk/prompt-engineering-guide-how-to-engineer-the-perfect-prompts/) -- [A Generic Framework for ChatGPT Prompt Engineering](https://medium.com/@thorbjoern.heise/a-generic-framework-for-chatgpt-prompt-engineering-7097f6513a0b) -- [Methods of prompt programming](https://generative.ink/posts/methods-of-prompt-programming/) - [Prompt Engineering 101](https://humanloop.com/blog/prompt-engineering-101) +- [Prompt Engineering by co:here](https://docs.cohere.ai/docs/prompt-engineering) +- [Prompt Engineering by Microsoft](https://microsoft.github.io/prompt-engineering/) +- [Prompt engineering davinci-003 on our own docs for automated support (Part I)](https://www.patterns.app/blog/2022/12/21/finetune-llm-tech-support/) +- [Prompt Engineering Guide: How to Engineer the Perfect Prompts](https://richardbatt.co.uk/prompt-engineering-guide-how-to-engineer-the-perfect-prompts/) +- [Prompt Engineering in GPT-3](https://www.analyticsvidhya.com/blog/2022/05/prompt-engineering-in-gpt-3/) +- [Prompt Engineering Template](https://docs.google.com/spreadsheets/d/1-snKDn38-KypoYCk9XLPg799bHcNFSBAVu2HVvFEAkA/edit#gid=0) +- [Prompt Engineering Topic by GitHub](https://github.com/topics/prompt-engineering) +- [Prompt Engineering: From Words to Art](https://www.saxifrage.xyz/post/prompt-engineering) +- [Prompt injection attacks against GPT-3](https://simonwillison.net/2022/Sep/12/prompt-injection/) +- [Prompt injection to read out the secret OpenAI API key](https://twitter.com/ludwig_stumpp/status/1619701277419794435?s=20&t=GtoMlmYCSt-UmvjqJVbBSA) +- [Prompting Methods with Language Models and Their Applications to Weak Supervision](https://snorkel.ai/prompting-methods-with-language-models-nlp/) +- [Prompts as Programming by Gwern](https://www.gwern.net/GPT-3#prompts-as-programming) +- [Reverse Prompt Engineering for Fun and (no) Profit](https://lspace.swyx.io/p/reverse-prompt-eng) +- [Simulators](https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators) +- [Start with an Instruction](https://beta.openai.com/docs/quickstart/start-with-an-instruction) +- [Talking to machines: prompt engineering & injection](https://artifact-research.com/artificial-intelligence/talking-to-machines-prompt-engineering-injection/) - [the Book - Fed Honeypot](https://fedhoneypot.notion.site/25fdbdb69e9e44c6877d79e18336fe05?v=1d2bf4143680451986fd2836a04afbf4) -- [Curtis64's set of prompt gists](https://gist.github.com/Curtis-64) +- [The ChatGPT Prompt Book](https://docs.google.com/presentation/d/17b_ocq-GL5lhV_bYSShzUgxL02mtWDoiw9xEroJ5m3Q/edit#slide=id.gc6f83aa91_0_79) +- [Using GPT-Eliezer against ChatGPT Jailbreaking](https://www.alignmentforum.org/posts/pNcFYZnPdXyL2RfgA/using-gpt-eliezer-against-chatgpt-jailbreaking) + # Lecture + Tutorial + Full tutorial and lecture coming soon! --- + Feel free to open a PR if you think something is missing here. Always welcome feedback and suggestions. Join our [Discord](https://discord.gg/SKgkVT8BGJ) From 06d09a60cd0111958094c19b16fcf651b9c44e80 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E5=B0=8F=E8=87=A3=E5=AD=90=E5=90=83=E5=A4=A7=E6=A9=99?= =?UTF-8?q?=E5=AD=90?= Date: Fri, 3 Feb 2023 18:07:27 +0800 Subject: [PATCH 2/2] add a few entries, unify URL format --- README.md | 79 ++++++++++++++++++++++++++++--------------------------- 1 file changed, 40 insertions(+), 39 deletions(-) diff --git a/README.md b/README.md index 8de12c8..cdd95b3 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,7 @@ This guide contains a set of papers, learning guides, and tools related to promp - Surveys / Overviews: - [A Survey for In-context Learning](https://arxiv.org/abs/2301.00234) (Dec 2022) + - [Towards Reasoning in Large Language Models: A Survey](https://arxiv.org/abs/2212.10403) (Dec 2022) - [Emergent Abilities of Large Language Models](https://arxiv.org/abs/2206.07682) (Jun 2022) - [A Taxonomy of Prompt Modifiers for Text-To-Image Generation](https://arxiv.org/abs/2204.13988) (Apr 2022) - [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://arxiv.org/abs/2107.13586) (Jul 2021) @@ -60,9 +61,9 @@ This guide contains a set of papers, learning guides, and tools related to promp - [Generated Knowledge Prompting for Commonsense Reasoning](https://arxiv.org/abs/2110.08387) (Oct 2021) - [Reframing Instructional Prompts to GPTk's Language](https://arxiv.org/abs/2109.07830) (Sep 2021) - [Design Guidelines for Prompt Engineering Text-to-Image Generative Models](https://arxiv.org/abs/2109.06977) (Sep 2021) - - [Making Pre-trained Language Models Better Few-shot Learners](https://aclanthology.org/2021.acl-long.295/) (Aug 2021) + - [Making Pre-trained Language Models Better Few-shot Learners](https://aclanthology.org/2021.acl-long.295) (Aug 2021) - [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://arxiv.org/abs/2104.08786) (April 2021) - - [BERTese: Learning to Speak to BERT](https://aclanthology.org/2021.eacl-main.316/) (April 2021) + - [BERTese: Learning to Speak to BERT](https://aclanthology.org/2021.eacl-main.316) (April 2021) - [The Power of Scale for Parameter-Efficient Prompt Tuning](https://arxiv.org/abs/2104.08691) (April 2021) - [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://arxiv.org/abs/2102.07350) (Feb 2021) - [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://arxiv.org/abs/2102.09690) (Feb 2021) @@ -76,7 +77,7 @@ This guide contains a set of papers, learning guides, and tools related to promp - [Investigating Prompt Engineering in Diffusion Models](https://arxiv.org/abs/2211.15462) (Nov 2022) - [Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language](https://arxiv.org/abs/2210.15157) (Oct 2022) - [Piloting Copilot and Codex: Hot Temperature, Cold Prompts, or Black Magic?](https://arxiv.org/abs/2210.14699) (Oct 2022) - - [Plot Writing From Scratch Pre-Trained Language Models](https://aclanthology.org/2022.inlg-main.5/) (July 2022) + - [Plot Writing From Scratch Pre-Trained Language Models](https://aclanthology.org/2022.inlg-main.5) (July 2022) - Collections: - [Chain-of-ThoughtsPapers](https://github.com/Timothyxxx/Chain-of-ThoughtsPapers) @@ -86,32 +87,32 @@ This guide contains a set of papers, learning guides, and tools related to promp ## Tools & Libraries #### (Sorted by Name) -- [AI Test Kitchen](https://aitestkitchen.withgoogle.com/) +- [AI Test Kitchen](https://aitestkitchen.withgoogle.com) - [betterprompt](https://github.com/krrishdholakia/betterprompt) -- [DreamStudio](https://beta.dreamstudio.ai/) -- [DUST](https://dust.tt/) -- [Dyno](https://trydyno.com/) -- [EveryPrompt](https://www.everyprompt.com/) +- [DreamStudio](https://beta.dreamstudio.ai) +- [DUST](https://dust.tt) +- [Dyno](https://trydyno.com) +- [EveryPrompt](https://www.everyprompt.com) - [GPT Index](https://github.com/jerryjliu/gpt_index) - [GPTTools](https://gpttools.com/comparisontool) - [hwchase17/adversarial-prompts](https://github.com/hwchase17/adversarial-prompts) - [Interactive Composition Explorer](https://github.com/oughtinc/ice) - [LangChain](https://github.com/hwchase17/langchain) -- [LearnGPT](https://www.learngpt.com/) -- [Lexica](https://lexica.art/) +- [LearnGPT](https://www.learngpt.com) +- [Lexica](https://lexica.art) - [Metaprompt](https://metaprompt.vercel.app/?task=gpt) - [OpenAI Playground](https://beta.openai.com/playground) - [OpenPrompt](https://github.com/thunlp/OpenPrompt) -- [Playground](https://playgroundai.com/) -- [Prompt Base](https://promptbase.com/) +- [Playground](https://playgroundai.com) +- [Prompt Base](https://promptbase.com) - [Prompt Engine](https://github.com/microsoft/prompt-engine) -- [Prompt Generator for OpenAI's DALL-E 2](http://dalle2-prompt-generator.s3-website-us-west-2.amazonaws.com/) -- [Promptable](https://promptable.ai/) +- [Prompt Generator for OpenAI's DALL-E 2](http://dalle2-prompt-generator.s3-website-us-west-2.amazonaws.com) +- [Promptable](https://promptable.ai) - [PromptInject](https://github.com/agencyenterprise/PromptInject) - [Prompts.ai](https://github.com/sevazhidkov/prompts-ai) - [PromptSource](https://github.com/bigscience-workshop/promptsource) - [Scale SpellBook](https://scale.com/spellbook) -- [sharegpt](https://sharegpt.com/) +- [sharegpt](https://sharegpt.com) - [ThoughtSource](https://github.com/OpenBioLink/ThoughtSource) - [Visual Prompt Builder](https://tools.saxifrage.xyz/prompt) @@ -123,66 +124,66 @@ This guide contains a set of papers, learning guides, and tools related to promp - [DiffusionDB](https://github.com/poloclub/diffusiondb) - [Midjourney Prompts](https://huggingface.co/datasets/succinctly/midjourney-prompts) - [P3 - Public Pool of Prompts](https://huggingface.co/datasets/bigscience/P3) -- [PartiPrompts](https://parti.research.google/) +- [PartiPrompts](https://parti.research.google) - [Real Toxicity Prompts](https://allenai.org/data/real-toxicity-prompts) - [Stable Diffusion Dataset](https://huggingface.co/datasets/Gustavosta/Stable-Diffusion-Prompts) -- [WritingPrompts](https://www.reddit.com/r/WritingPrompts/) +- [WritingPrompts](https://www.reddit.com/r/WritingPrompts) ## Blog, Guides, Tutorials and Other Readings #### (Sorted by Name) -- [3 Principles for prompt engineering with GPT-3](https://www.linkedin.com/pulse/3-principles-prompt-engineering-gpt-3-ben-whately/) +- [3 Principles for prompt engineering with GPT-3](https://www.linkedin.com/pulse/3-principles-prompt-engineering-gpt-3-ben-whately) - [A beginner-friendly guide to generative language models - LaMBDA guide](https://aitestkitchen.withgoogle.com/how-lamda-works) -- [A Complete Introduction to Prompt Engineering for Large Language Models](https://www.mihaileric.com/posts/a-complete-introduction-to-prompt-engineering/) +- [A Complete Introduction to Prompt Engineering for Large Language Models](https://www.mihaileric.com/posts/a-complete-introduction-to-prompt-engineering) - [A Generic Framework for ChatGPT Prompt Engineering](https://medium.com/@thorbjoern.heise/a-generic-framework-for-chatgpt-prompt-engineering-7097f6513a0b) - [AI Content Generation](https://www.jonstokes.com/p/ai-content-generation-part-1-machine) - [Awesome ChatGPT Prompts](https://github.com/f/awesome-chatgpt-prompts) -- [Best 100+ Stable Diffusion Prompts](https://mpost.io/best-100-stable-diffusion-prompts-the-most-beautiful-ai-text-to-image-prompts/) +- [Best 100+ Stable Diffusion Prompts](https://mpost.io/best-100-stable-diffusion-prompts-the-most-beautiful-ai-text-to-image-prompts) - [Best practices for prompt engineering with OpenAI API](https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api) +- [ChatGPT, AI and GPT-3 Apps and use cases](https://gpt3demo.com) - [CMU Advanced NLP 2022: Prompting](https://youtube.com/watch?v=5ef83Wljm-M&feature=shares) - [Curtis64's set of prompt gists](https://gist.github.com/Curtis-64) - [DALL·E 2 Prompt Engineering Guide](https://docs.google.com/document/d/11WlzjBT0xRpQhP9tFMtxzd0q6ANIdHPUBkMV-YB043U/edit#) -- [DALLE Prompt Book](https://dallery.gallery/the-dalle-2-prompt-book/) +- [DALLE Prompt Book](https://dallery.gallery/the-dalle-2-prompt-book) - [Exploiting GPT-3 Prompts](https://twitter.com/goodside/status/1569128808308957185) -- [Exploring Prompt Injection Attacks](Exploring Prompt Injection Attacks) -- [Extrapolating to Unnatural Language Processing with GPT-3's In-context Learning: The Good, the Bad, and the Mysterious](http://ai.stanford.edu/blog/in-context-learning/) -- [Generative AI with Cohere: Part 1 - Model Prompting](https://txt.cohere.ai/generative-ai-part-1/) +- [Exploring Prompt Injection Attacks](https://research.nccgroup.com/2022/12/05/exploring-prompt-injection-attacks) +- [Extrapolating to Unnatural Language Processing with GPT-3's In-context Learning: The Good, the Bad, and the Mysterious](http://ai.stanford.edu/blog/in-context-learning) +- [Generative AI with Cohere: Part 1 - Model Prompting](https://txt.cohere.ai/generative-ai-part-1) - [Giving GPT-3 a Turing Test](https://lacker.io/ai/2020/07/06/giving-gpt-3-a-turing-test.html) - [GPT3 and Prompts: A quick primer](https://buildspace.so/notes/intro-to-gpt3-prompts) -- [How to Draw Anything](https://andys.page/posts/how-to-draw/) -- [How to get images that don't suck](https://www.reddit.com/r/StableDiffusion/comments/x41n87/how_to_get_images_that_dont_suck_a/) -- [How to write good prompts](https://andymatuschak.org/prompts/) +- [How to Draw Anything](https://andys.page/posts/how-to-draw) +- [How to get images that don't suck](https://www.reddit.com/r/StableDiffusion/comments/x41n87/how_to_get_images_that_dont_suck_a) +- [How to write good prompts](https://andymatuschak.org/prompts) - [Language Models and Prompt Engineering: Systematic Survey of Prompting Methods in NLP](https://youtube.com/watch?v=OsbUfL8w-mo&feature=shares) -- [Learn Prompting](https://learnprompting.org/) -- [Methods of prompt programming](https://generative.ink/posts/methods-of-prompt-programming/) +- [Learn Prompting](https://learnprompting.org) +- [Methods of prompt programming](https://generative.ink/posts/methods-of-prompt-programming) - [Mysteries of mode collapse](https://www.lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-of-mode-collapse) - [NLP for Text-to-Image Generators: Prompt Analysis](https://heartbeat.comet.ml/nlp-for-text-to-image-generators-prompt-analysis-part-1-5076a44d8365) - [Notes for Prompt Engineering by sw-yx](https://github.com/sw-yx/ai-notes) -- [Pretrain, Prompt, Predict - A New Paradigm for NLP](http://pretrain.nlpedia.ai/) -- [Prompt Engineering 101 - Introduction and resources](https://www.linkedin.com/pulse/prompt-engineering-101-introduction-resources-amatriain/) +- [Pretrain, Prompt, Predict - A New Paradigm for NLP](http://pretrain.nlpedia.ai) +- [Prompt Engineering 101 - Introduction and resources](https://www.linkedin.com/pulse/prompt-engineering-101-introduction-resources-amatriain) - [Prompt Engineering 101: Autocomplete, Zero-shot, One-shot, and Few-shot prompting](https://youtube.com/watch?v=v2gD8BHOaX4&feature=shares) - [Prompt Engineering 101](https://humanloop.com/blog/prompt-engineering-101) - [Prompt Engineering by co:here](https://docs.cohere.ai/docs/prompt-engineering) -- [Prompt Engineering by Microsoft](https://microsoft.github.io/prompt-engineering/) -- [Prompt engineering davinci-003 on our own docs for automated support (Part I)](https://www.patterns.app/blog/2022/12/21/finetune-llm-tech-support/) -- [Prompt Engineering Guide: How to Engineer the Perfect Prompts](https://richardbatt.co.uk/prompt-engineering-guide-how-to-engineer-the-perfect-prompts/) -- [Prompt Engineering in GPT-3](https://www.analyticsvidhya.com/blog/2022/05/prompt-engineering-in-gpt-3/) +- [Prompt Engineering by Microsoft](https://microsoft.github.io/prompt-engineering) +- [Prompt engineering davinci-003 on our own docs for automated support (Part I)](https://www.patterns.app/blog/2022/12/21/finetune-llm-tech-support) +- [Prompt Engineering Guide: How to Engineer the Perfect Prompts](https://richardbatt.co.uk/prompt-engineering-guide-how-to-engineer-the-perfect-prompts) +- [Prompt Engineering in GPT-3](https://www.analyticsvidhya.com/blog/2022/05/prompt-engineering-in-gpt-3) - [Prompt Engineering Template](https://docs.google.com/spreadsheets/d/1-snKDn38-KypoYCk9XLPg799bHcNFSBAVu2HVvFEAkA/edit#gid=0) - [Prompt Engineering Topic by GitHub](https://github.com/topics/prompt-engineering) - [Prompt Engineering: From Words to Art](https://www.saxifrage.xyz/post/prompt-engineering) -- [Prompt injection attacks against GPT-3](https://simonwillison.net/2022/Sep/12/prompt-injection/) +- [Prompt injection attacks against GPT-3](https://simonwillison.net/2022/Sep/12/prompt-injection) - [Prompt injection to read out the secret OpenAI API key](https://twitter.com/ludwig_stumpp/status/1619701277419794435?s=20&t=GtoMlmYCSt-UmvjqJVbBSA) -- [Prompting Methods with Language Models and Their Applications to Weak Supervision](https://snorkel.ai/prompting-methods-with-language-models-nlp/) +- [Prompting Methods with Language Models and Their Applications to Weak Supervision](https://snorkel.ai/prompting-methods-with-language-models-nlp) - [Prompts as Programming by Gwern](https://www.gwern.net/GPT-3#prompts-as-programming) - [Reverse Prompt Engineering for Fun and (no) Profit](https://lspace.swyx.io/p/reverse-prompt-eng) - [Simulators](https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators) - [Start with an Instruction](https://beta.openai.com/docs/quickstart/start-with-an-instruction) -- [Talking to machines: prompt engineering & injection](https://artifact-research.com/artificial-intelligence/talking-to-machines-prompt-engineering-injection/) +- [Talking to machines: prompt engineering & injection](https://artifact-research.com/artificial-intelligence/talking-to-machines-prompt-engineering-injection) - [the Book - Fed Honeypot](https://fedhoneypot.notion.site/25fdbdb69e9e44c6877d79e18336fe05?v=1d2bf4143680451986fd2836a04afbf4) - [The ChatGPT Prompt Book](https://docs.google.com/presentation/d/17b_ocq-GL5lhV_bYSShzUgxL02mtWDoiw9xEroJ5m3Q/edit#slide=id.gc6f83aa91_0_79) - [Using GPT-Eliezer against ChatGPT Jailbreaking](https://www.alignmentforum.org/posts/pNcFYZnPdXyL2RfgA/using-gpt-eliezer-against-chatgpt-jailbreaking) - # Lecture + Tutorial Full tutorial and lecture coming soon!