Resources

Large language model (LLM)

​​

Open LLMs List

https://github.com/eugeneyan/open-llms

Introducing Open Platform for Enterprise AI
https://www.youtube.com/watch?v=Q5uTvmv5fZU&t=4s

Open Platform for Enterprise AI

https://opea.dev/

https://github.com/opea-project

​

Nvidia

https://www.nvidia.com/en-eu/ai-data-science/generative-ai/

​

AWS Bedrock

https://aws.amazon.com/bedrock/

​

Google Gemma

https://ai.google.dev/gemma

Google Gemini

https://deepmind.google/technologies/gemini/#introduction

​

Mistral AI brings a strong open generative models to the developers, along with efficient ways to deploy and customise them for production.

https://mistral.ai

​

Meta Llama
https://ai.meta.com/llama/
 

Alibaba Cloud provides Tongyi Qianwen (Qwen)
https://www.alibabacloud.com/en/solutions/generative-ai/qwen?_p_lc=1

​​​

What’s before GPT-4? A deep dive into ChatGPT

https://medium.com/digital-sense-ai/whats-before-gpt-4-a-deep-dive-into-chatgpt-dfce9db49956

​

GPT-4 Training process

Like previous GPT models, the GPT-4 base model was trained to predict the next word in a document, and was trained using publicly available data (such as internet data) as well as data we’ve licensed. The data is a web-scale corpus of data including correct and incorrect solutions to math problems, weak and strong reasoning, self-contradictory and consistent statements, and representing a great variety of ideologies and ideas.

So when prompted with a question, the base model can respond in a wide variety of ways that might be far from a user’s intent. To align it with the user’s intent within guardrails, we fine-tune the model’s behavior using reinforcement learning with human feedback (RLHF).

Note that the model’s capabilities seem to come primarily from the pre-training process—RLHF does not improve exam performance (without active effort, it actually degrades it). But steering of the model comes from the post-training process—the base model requires prompt engineering to even know that it should answer the questions.

https://openai.com/research/gpt-4

​​​

BERT
https://github.com/google-research/bert

AlephBERT

https://github.com/OnlpLab/AlephBERT
https://arxiv.org/pdf/2104.04052.pdf

Multi-language Aspects
How Language-Neutral is Multilingual BERT?​
https://arxiv.org/pdf/1911.03310.pdf

AraBERT: Transformer-based Model for Arabic Language Understanding
https://arxiv.org/pdf/2003.00104.pdf
 
ELMo
https://allennlp.org/elmo

 

LaBSE - Language-agnostic BERT sentence embedding model supporting 109 languages.

https://tfhub.dev/google/LaBSE/2

​

LaBSE model to PyTorch. It can be used to map 109 languages to a shared vector space.
https://huggingface.co/sentence-transformers/LaBSE

​

Claude is a large language model (LLM) built by Anthropic.
It's trained to be a helpful assistant in a conversational tone.

https://docs.anthropic.com/claude/docs/getting-started-with-claude

​

Jais - open-source Arabic Large Language Model (LLM)

https://huggingface.co/core42/jais-13b/tree/main

https://inceptioniai.org/jais/

​

Falcon LLM - Falcon 40B was the world’s top-ranked open-source AI model when launched

https://falconllm.tii.ae/falcon.html​

​

Mamba: Linear-Time Sequence Modeling with Selective State Spaces

https://huggingface.co/TRI-ML/mamba-7b-rw
https://huggingface.co/papers/2312.00752

https://www.together.ai/blog/mamba-3b-slimpj

​