Server
Requirements
Accept the code of conduct and follow the server setup guide to setup the development environment.
Gateway
The Gateway is a WebSocket server that is responsible for listening and emitting events.
For documentation, head over to the Discord docs as our own documentation is not written yet.
If you want to work on a feature, please comment on the corresponding issue or open a issue so so nobody implements something twice.
For the WebSocket, we use the ws package and we'll write our own packet handler for the individual opcodes and events.
API
The API is a HTTP REST server that process requests and manipulates the database.
You can find the api documentation here.
You can find the Roadmap overview here.
Every route has its own issue.
If you want to work on a feature please comment on the corresponding issue or write us on our development server so we can assign and discuss it and nobody implements something twice.
Structure
You can find the API directory in the fosscord-server Github repository.
Inside it you can find:
Translation
We use i18next to manage translation/localization in some API Responses.
The .json
language files are located in /api/locales/
and are separated by namespaces.
Source code
We use TypeScript (JavaScript with types).
The .ts
source files are located in /api/src/
and will be compiled to .js
in the /api/dist/
directory.
Middlewares
All Express Middlewares are in /api/src/middlewares/
and need to be manually loaded by /api/src/Server.ts
.
Routes
All Express Router routes are in /api/src/routes/
and are automatically registered based on the file structure.
Models
All database TypeORM entities are located in /util/src/entities
Util
All Utility functions are in the directory /src/util/
and in @fosscord/util
Configuration
Philosophy
Every fosscord server instance should be completely configurable in every way, without the need to change the source code.
The config should have reasonable defaults similar to discord.
Only in special cases it should require a third party config value.
The config should be changeable over the admin dashboard and update in realtime without the need to restart the servers.
The very first time the server starts, it saves to default config in the database. The next start it will load the config from the database.
Example
You should not get()
the Config in the root of your file and it instead load the config every time you access a value.
Import Config
from fosscord-server-util:
// at the top of the file import the Config file from /src/util/Config.ts
import { Config } from "@fosscord-server-util";
Access the Config in your route:
router.get("/", (req: Request, res: Response) => {
// call Config.get() to get the whole config object and then just access the property you want
const { allowNewRegistration } = Config.get().register;
});
Config.get()
returns the current config object and is not expensive at all
Extending
The default Config is located in server-util /src/util/Config.ts
and exports a interface DefaultOptions
and a const DefaultOptions
object with reasonable default values.
To add your own values to the config, add the properties to the interface
with corresponding types and add default values to const DefaultOptions
.
Also you don't need to worry about updating "old config versions", because new values will automatically be synced with the database.
Note, however, that if the database already has a default value it won't update it.
Routes
All routes are located in the directory /src/routes/
and are loaded on start by a the lambert-server package.
The HTTP API path is generated automatically based on the folder structure, so it is important that you name your files accordingly.
If you want to use URL Params like :id
in e.g. /users/:id
you need to use #
instead of :
for the folder/filename, because of file naming issues on windows.
index.ts
files won't serve /api/index
and instead alias the parent folder e.g. /api/
Your file needs to default export a express.Router():
import { Router } from express;
const router = Router();
export default router;
Now you can just use any regular express function on the router variable e.g:
router.get("/", (req, res) => {});
router.post("/", (req, res) => {});
router.get("/members", (req, res) => {});
Authentication
Every request must contain the authorization header except the /login
and /register
route.
You can add additional non-auth routes in /src/middlewares/Authentication.ts
To access the user id for the current request use req.user_id
Body Validation
We use a custom body validation logic from lambert-server to check if the JSON body is valid.
To import the function from /src/util/instanceOf.ts
use:
import { check } from "/src/util/instanceOf";
Now you can use the middleware check
for your routes by calling check with your Body Schema.
router.post("/", check(...), (req,res) => {});
Schema
A Schema is a Object Structure with key-value objects that checks if the supplied body is an instance of the specified class.
{ id: String, roles: [String] }
Notice if you use e.g. BigInt even if you can't supply it with JSON, it will automatically convert the supplied JSON number/string to a BigInt.
Also if you want to check for an array of, just put the type inside []
.
Optional Parameter
You can specify optional parameters if you prefix the key with a $
(dollar sign) e.g.: { $captcha: String }
, this will make the captcha property in the body optional.
Limit String length
Additionally import the class Length
from instanceOf and specify the type by making a new Length
Object taking following parameters:
import { Length } from "/src/util/instanceOf";
const min = 2;
const max = 32;
const type = String;
{
username: new Length(min, max, type);
}
this will limit the maximum string/number/array length to the min
and max
value.
Example
import { check, Length } from "/src/util/instanceOf";
const SCHEMA = {
username: new Length(2, 32, String),
age: Number,
$posts: [{ title: String }],
};
app.post("/", check(SCHEMA), (req, res) => {});
Throw Errors
If the body validation fails it will automatically throw an error.
The errors
structure is a key-value Object describing what field contained the error:
{
"code": 50035,
"message": "Invalid Form Body",
"errors": {
"email": {
"_errors": [
{
"message": "Email is already registered",
"code": "EMAIL_ALREADY_REGISTERED"
}
]
},
"username": {
"_errors": [
{
"message": "Must be between 2 - 32 in length",
"code": "BASE_TYPE_BAD_LENGTH"
}
]
}
}
}
To manually throw a FieldError
import FieldErrors
import { FieldErrors } from /src/iltu / instanceOf;
To make sure your errors are understood in all languages translate it with i18next and req.t
.
So after you have checked the field is invalid throw the FieldErrors
.
throw FieldErrors(( login: { message: req.t("auth:login.INVALID_LOGIN"), code: "INVALID_LOGIN" }});
Database
Philosophy
The instance hoster should be able to use any database they want for their specific size and purpose.
That is why we use typeorm for database entities (models) for every data structure we use, because typeorm supports many different database engines.
We use strings for all ids and bitfields (Tho when working with bitfields we convert it to BigInts and pass it to the utility BitField
class).
General
Have a look at the typeorm documentation to get familiar with it or watch this tutorial.
TypeORM supports MySQL, MariaDB, Postgres, CockroachDB, SQLite, Microsoft SQL Server, Oracle, SAP Hana, sql.js.
Getting Started
Import the entity you want to select, manipulate, delete or insert from @fosscord/util
List of all entities: Application, Attachment, AuditLog, Ban, BaseClass, Channel, Config, ConnectedAccount, Emoji, Guild, Invite, Member, Message, RateLimit, ReadState, Recipient, Relationship, Role, Sticker, Team, TeamMember, Template, User, VoiceState, Webhook
Example database query
import { Guild } from "fosscord-server-util";
await new Guild({ ... }).save(); // inserts a new guild or updates it if it already exists
const guild = await Guild.findOne({ id: "23948723947932" }).exec(); // searches for a guild
await Guild.delete({ owner_id: "34975309473" }) // deletes all guilds of the specific owner
Entities
The typeorm database entities are located in util/src/entities/
.
To add your own database entity, create a new file, export the model and import/export it in util/src/entities/index.ts
.
Example entity
@Entity("users")
export class User extends BaseClass {
// id column is automatically added by BaseClass
@Column()
username: string;
@JoinColumn({ name: "connected_account_ids" })
@OneToMany(
() => ConnectedAccount,
(account: ConnectedAccount) => account.user
)
connected_accounts: ConnectedAccount[];
static async getPublicUser(user_id: string, opts?: FindOneOptions<User>) {
return await User.findOneOrFail(
{ id: user_id },
{
...opts,
select: [...PublicUserProjection, ...(opts?.select || [])],
}
);
}
}
Emit Events
Most Routes modify the database and therefore need to inform the clients with events for data changes.
Events are either stored locally if the server was started through the bundle or in RabbitMQ and are distributed to the gateway servers.
You can find all events on the discord docs page and in util/src/interfaces/Event.ts
.
To emit an event import the emitEvent
function from @fosscord/util
import { emitEvent } from "../../../util/Event";
You need to specify whom you want to send the event to, to do that either pass guild_id
, user_id
or channel_id
.
Additionally you need to set the eventname e.g. GUILD_DELETE
.
{
guild_id?: bigint; // specify this if this event should be sent to all guild members
channel_id?: bigint; // specify this if this event should be sent to all channel members
user_id?: bigint; // specify this if this event should be sent to the specific user
event: string; // the EVENTNAME, you can find all gateway event names in the @fosscord/util Events file
data?: any; // event payload data
}
For easy intellisense, annotate the parameter with the corresponding Event interface from @fosscord/util
:
import { GuildDeleteEvent } from "@fosscord/util";
emitEvent({...} as GuildDeleteEvent);
Example
Putting it all together:
await emitEvent({
user_id: "3297349345345874",
event: "GUILD_DELETE",
data: {
id: "96784598743975349",
},
} as GuildDeleteEvent);
Permissions
To get the permission for a guild member import the getPermission
from fosscord-server-util
.
import { getPermission } from "fosscord-server-util";
The first argument is the user_id the second the guild_id and the third an optional channel_id.
const permissions = await getPermission(user_id: string, guild_id: string, channel_id?: string)
const permissions = await getPermission("106142653265366125", "4061326832657368175")
Example
const perms = await getPermission(req.userid, guild_id);
// preferred method: Use this if you want to check if a user lacks a certain permission and abort the operation
perms.hasThrow("MANAGE_GUILD") // will throw an error if the users lacks the permission
if (perms.has("MANAGE_GUILD")) {
...
}