Awesome List Updates on May 13, 2024

8 awesome lists updated today.

🏠 Home · 🔍 Search · 🔥 Feed · 📮 Subscribe · ❤️ Sponsor

1. Free for Dev

Design and UI

• movingpencils.com — Fast, browser-based vector editor. Completely free.

2. Awesome Playcanvas

Browser Games

• Build Land - Grow your island by collecting resources.

Product Configurators

• Norqain - Watch configurator.

3. Awesome Vite

Templates / React

• React Tanning (⭐3) - Starter template with React + Vite + TypeScript + TanStack + Tailwind.

PHP / Community

• PHP-Vite (⭐20) - Integration for PHP, Composer package, no framework dependencies.

Apps/Websites / Community

• Dataviz Explorer (⭐15) - A tool for large CSV, database, and real-time visualization with interactive plots using Highcharts, Chart.js, React, JavaScript, Material UI and GitHub Actions with Coverage Report.

4. Awesome Terraform

Self-Hosted Registries / Miscellaneous

• terralist (⭐307) - Terraform Private Registry for modules and providers manageable from a REST API.

5. Awesome Azure Openai Llm

Vector Database Comparison

• Milvus (A cloud-native vector database) Embedded git (⭐29k) [Sep 2019]: Alternative option to replace PineCone and Redis Search in OSS. It offers support for multiple languages, addresses the limitations of RedisSearch, and provides cloud scalability and high reliability with Kubernetes.

• Pinecone: A fully managed cloud Vector Database. Commercial Product [Jan 2021]

• Weaviate (⭐11k): Store both vectors and data objects. [Jan 2021]

• Chroma (⭐14k): Open-source embedding database [Oct 2022]

• Qdrant (⭐20k): Written in Rust. Qdrant (read: quadrant) [May 2020]

• Redis extension for vector search, RedisVL (⭐200): Redis Vector Library (RedisVL) [Nov 2022]

• A SQLite extension for efficient vector search, based on Faiss! (⭐1.7k) [Jan 2023]

Microsoft Azure OpenAI relevant LLM Framework / Lucene based search engine with OpenAI Embedding

• SAMMO (⭐301): A general-purpose framework for prompt optimization. ref [April 2024]

LangChain chain type: Chains & Summarizer / DSPy optimizer

• Chains ref (⭐0)
  • SimpleSequentialChain: A sequence of steps with single input and output. Output of one step is input for the next.
  • SequentialChain: Like SimpleSequentialChain but handles multiple inputs and outputs at each step.
  • MultiPromptChain: Routes inputs to specialized sub-chains based on content. Ideal for different prompts for different tasks.

• Summarizer
  • stuff: Sends everything at once in LLM. If it's too long, an error will occur.
  • map_reduce: Summarizes by dividing and then summarizing the entire summary.
  • refine: (Summary + Next document) => Summary
  • map_rerank: Ranks by score and summarizes to important points.

Prompt Engineering / Prompt Template Language

• Many-Shot In-Context Learning: Transitioning from few-shot to many-shot In-Context Learning (ICL) can lead to significant performance gains across a wide variety of generative and discriminative tasks [17 Apr 2024]

Prompt Guide & Leaked prompts / Prompt Template Language

• Awesome ChatGPT Prompts (⭐108k) [Dec 2022]

• Awesome Prompt Engineering (⭐3.6k) [Feb 2023]

• Awesome-GPTs-Prompts (⭐4.9k) [Jan 2024]

• Leaked prompts of GPTs (⭐28k) [Nov 2023] and Agents (⭐7.9k) [Nov 2023]

Memory Optimization / Llama Finetuning

• Transformer cache key-value tensors of context tokens into GPU memory to facilitate fast generation of the next token. However, these caches occupy significant GPU memory. The unpredictable nature of cache size, due to the variability in the length of each request, exacerbates the issue, resulting in significant memory fragmentation in the absence of a suitable memory management mechanism.

• To alleviate this issue, PagedAttention was proposed to store the KV cache in non-contiguous memory spaces. It partitions the KV cache of each sequence into multiple blocks, with each block containing the keys and values for a fixed number of tokens.

• PagedAttention : vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention, 24x Faster LLM Inference doc. ref [12 Sep 2023]

  
  • PagedAttention for a prompt “the cat is sleeping in the kitchen and the dog is”. Key-Value pairs of tensors for attention computation are stored in virtual contiguous blocks mapped to non-contiguous blocks in the GPU memory.

• TokenAttention (⭐2.2k) an attention mechanism that manages key and value caching at the token level. git (⭐2.2k) [Jul 2023]

• Flash Attention: [cnt] [27 May 2022] / FlashAttention-2: [cnt] [17 Jul 2023]: An method that reorders the attention computation and leverages classical techniques (tiling, recomputation). Instead of storing each intermediate result, use kernel fusion and run every operation in a single kernel in order to avoid memory read/write overhead. git (⭐13k) -> Compared to a standard attention implementation in PyTorch, FlashAttention-2 can be up to 9x faster / FlashAttention-3 [11 Jul 2024]

Other techniques and LLM patterns / Llama Finetuning

• Better & Faster Large Language Models via Multi-token Prediction: Suggest that training language models to predict multiple future tokens at once [30 Apr 2024]

OpenAI's Roadmap and Products / OpenAI's plans according to Sam Altman

• Model Spec: Desired behavior for the models in the OpenAI API and ChatGPT ref [8 May 2024] ref: takeaway

Context constraints / GPT series release date

• Introducing 100K Context Windows: hundreds of pages, Around 75,000 words; [11 May 2023] demo Anthropic Claude

• “Needle in a Haystack” Analysis [21 Nov 2023]: Context Window Benchmarks; Claude 2.1 (200K Context Window) vs GPT-4 (⭐1.4k); Long context prompting for Claude 2.1 adding just one sentence, “Here is the most relevant sentence in the context:”, to the prompt resulted in near complete fidelity throughout Claude 2.1’s 200K context window. [6 Dec 2023]

• Rotary Positional Embedding (RoPE): [cnt] / ref / doc [20 Apr 2021]
  • How is this different from the sinusoidal embeddings used in "Attention is All You Need"?
    1. Sinusoidal embeddings apply to each coordinate individually, while rotary embeddings mix pairs of coordinates
    2. Sinusoidal embeddings add a cos or sin term, while rotary embeddings use a multiplicative factor.
    3. Rotary embeddings are applied to positional encoding to K and V, not to the input embeddings.

• Lost in the Middle: How Language Models Use Long Contexts: [cnt] [6 Jul 2023]
  1. Best Performace when relevant information is at beginning
  2. Too many retrieved documents will harm performance
  3. Performacnce decreases with an increase in context

• Structured Prompting: Scaling In-Context Learning to 1,000 Examples: [cnt] [13 Dec 2022]
  1. Microsoft's Structured Prompting allows thousands of examples, by first concatenating examples into groups, then inputting each group into the LM. The hidden key and value vectors of the LM's attention modules are cached. Finally, when the user's unaltered input prompt is passed to the LM, the cached attention vectors are injected into the hidden layers of the LM.
  2. This approach wouldn't work with OpenAI's closed models. because this needs to access [keys] and [values] in the transformer internals, which they do not expose. You could implement yourself on OSS ones. cite [07 Feb 2023]

• Ring Attention: [cnt]: 1. Ring Attention, which leverages blockwise computation of self-attention to distribute long sequences across multiple devices while overlapping the communication of key-value blocks with the computation of blockwise attention. 2. Ring Attention can reduce the memory requirements of Transformers, enabling us to train more than 500 times longer sequence than prior memory efficient state-of-the-arts and enables the training of sequences that exceed 100 million in length without making approximations to attention. 3. we propose an enhancement to the blockwise parallel transformers (BPT) framework. git (⭐595) [3 Oct 2023]

• LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning. With only four lines of code modification, the proposed method can effortlessly extend existing LLMs' context window without any fine-tuning. [2 Jan 2024]

• Giraffe: Adventures in Expanding Context Lengths in LLMs. A new truncation strategy for modifying the basis for the position encoding. ref [2 Jan 2024]

• Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention. The Infini-attention incorporates a compressive memory into the vanilla attention mechanism. Integrate attention from both local and global attention. [10 Apr 2024]

Trustworthy, Safe and Secure LLM / GPT series release date

• Trustworthy LLMs: [cnt]: Comprehensive overview for assessing LLM trustworthiness; Reliability, safety, fairness, resistance to misuse, explainability and reasoning, adherence to social norms, and robustness. [10 Aug 2023]

GPT for Domain Specific / GPT series release date

• TimeGPT: The First Foundation Model for Time Series Forecasting git (⭐2.9k) [Mar 2023]

• BioGPT: [cnt]: Generative Pre-trained Transformer for Biomedical Text Generation and Mining git (⭐4.3k) [19 Oct 2022]

• MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers [27 Nov 2023]

• BloombergGPT: A Large Language Model for Finance [30 Mar 2023]

• Galactica: A Large Language Model for Science [16 Nov 2022]

• EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domain [30 Jan 2024]

• SaulLM-7B: A pioneering Large Language Model for Law [6 Mar 2024]

• Huggingface StarCoder: A State-of-the-Art LLM for Code: git [May 2023]

• Code Llama: Built on top of Llama 2, free for research and commercial use. ref / git (⭐16k) [24 Aug 2023]

• Devin AI: Devin is an AI software engineer developed by Cognition AI [12 Mar 2024]

• OpenDevin: an open-source project aiming to replicate Devin [Mar 2024]

• FrugalGPT: LLM with budget constraints, requests are cascaded from low-cost to high-cost LLMs. git (⭐165) [9 May 2023]

MLLM (multimodal large language model) / GPT series release date

• Vision capability to a LLM ref [22 Aug 2023]

  • The model has three sub-models:

    1. A model to obtain image embeddings
    2. A text model to obtain text embeddings
    3. A model to learn the relationships between them

  • This is analogous to adding vision capability to a LLM.

    

Generative AI Landscape / GPT series release date

• The Generative AI Revolution: Exploring the Current Landscape : doc [28 Jun 2023]

• Diffusion Models vs. GANs vs. VAEs: Comparison of Deep Generative Models [12 May 2023]

LLM Materials for East Asian Languages / Japanese

• AI事業者ガイドライン [Apr 2024]

LLM Materials for East Asian Languages / Korean

• Machine Learning Study 혼자 해보기 (⭐2.6k) [Sep 2018]

• LangChain 한국어 튜토리얼 (⭐922) [Feb 2024]

Agents: AutoGPT and Communicative Agents / Tool use: LLM to Master APIs

• Gorilla: An API store for LLMs: [cnt]: Gorilla: Large Language Model Connected with Massive APIs git (⭐11k) [24 May 2023]

  1. Used GPT-4 to generate a dataset of instruction-api pairs for fine-tuning Gorilla.

  2. Used the abstract syntax tree (AST) of the generated code to match with APIs in the database and test set for evaluation purposes.

  Another user asked how Gorilla compared to LangChain; Patil replied: LangChain is a terrific project that tries to teach agents how to use tools using prompting. Our take on this is that prompting is not scalable if you want to pick between 1000s of APIs. So Gorilla is a LLM that can pick and write the semantically and syntactically correct API for you to call! A drop in replacement into LangChain! cite [04 Jul 2023]

• Meta: Toolformer: [cnt]: Language Models That Can Use Tools, by MetaAI git (⭐1.9k) [9 Feb 2023]

• ToolLLM: [cnt]: : Facilitating Large Language Models to Master 16000+ Real-world APIs git (⭐4.7k) [31 Jul 2023]

Section 11: Datasets for LLM Training / OSS Alternatives for OpenAI Code Interpreter (aka. Advanced Data Analytics)

• MS MARCO Web Search (⭐301): A large-scale information-rich web dataset, featuring millions of real clicked query-document labels [Apr 2024]

6. Awesome Rust

Applications / Utilities

• Mobslide (⭐342) - Desktop application that turns your smartphone into presentation remote controller.

• Vibe (⭐619) - Transcribe audio or video in every language on every platform.

7. Awesome Angular

RxJS / Google Developer Experts

• ngx-observable-lifecycle (⭐34) - Library for observing the lifecycle of an (ivy compiled) angular component.

• rx-sandbox (⭐172) - Marble diagram DSL based test suite for RxJS.

• observer-spy (⭐375) - This library makes RxJS Observables testing easy!

• ngx-operators (⭐136) - RxJS operators for Angular.

• rxjs-toolbox (⭐10) - Set of custom operators and handy factory functions for RxJS.

• subscribable-things (⭐42) - A collection of reactive wrappers for various browser APIs.

• ngx-rxjs-zone-scheduler (⭐32) - A library for Angular providing RxJS schedulers to run some kind of work inside or outside of NgZone.

• rxjs-broker (⭐24) - An RxJS message broker for WebRTC DataChannels and WebSockets.

• rxjs-insights (⭐359) - See through the observables.

8. Awesome Zsh Plugins

Plugins / superconsole - Windows-only

• brave (⭐0) - Manages Brave profiles. With this plugin, you can start the Brave Browser with a specific user profile by using the brave command followed by the profile's name. The plugin also implements autocompletion for the profile names so you won't have to type the entire profile name manually.

• explain-shell (brokentoaster) (⭐2) - Uses lynx to look up the current command line on explainshell.com.

• explain-shell (gmatheu) (⭐27) - Opens commands on explainshell.com.

• fixnumpad-osx (⭐3) - Enables numpad keys of Apple keyboards to be recognized in ZSH.

• goenv (cda0) (⭐0) - Plugin for installing, updating and loading goenv.

• ollama (⭐11) - Integrates the OLLAMA AI model with fzf (⭐63k) to provide intelligent command suggestions based on user input requirements.

• raspberryPi4Temperature (⭐0) - Puts the Raspberry Pi temperature into a spaceship prompt segment

• ros2-env (⭐0) - Manage ROS 2 environment and workspaces.

Completions / superconsole - Windows-only

• tofu (⭐0) - Autoloads tab completions for tofu.

Themes / superconsole - Windows-only

• candy-fantasy (⭐0) - Modified version of Candy Kingdowm (⭐172k)theme.

• Prev: May 14, 2024
• Next: May 12, 2024