sani7/2023-Webservices_And_Applications
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'main' into TFTP

Browse Source
This commit is contained in:
Sander Speetjens
2023-11-13 12:27:58 +01:00
parent ec1bff85b1 3f286df23f
commit 4c80601d4b
5 changed files with 1198 additions and 532 deletions
Download Patch File Download Diff File Expand all files Collapse all files

116
docs/llfs.md
View File

@@ -1,17 +1,52 @@
# LLFS (Linked List File System)
## Introduction
The llfs filesystem can be generated using the mkllfss utility.
It is a simple filesystem that uses a linked list to store files.
The filesystem is stored in a single c file (`llfs_data.c`), which can be compiled and read by the llfs library.
The llfs filesystem is a flat filesystem, meaning that it does not support directories.
The llfs filesystem can be generated using the [mkllfs](mkllfs.md) utility.
The resulting C file encapsulates the filesystem data within a linked list which can be used by the llfs library.
As a flat filesystem, llfs lacks support for directories.
Using the llfs API, information about the files in the filesystem can be retrieved,
and the files can be read using direct memory access.
Alternatively, the POSIX file functions can be used.
But this is more resource intensive, as data must be copied before using it.
It's essential to note that the llfs filesystem operates in a read-only mode,
restricting operations solely to read functions.
## Table of contents
- [Introduction](#introduction)
- [Table of contents](#table-of-contents)
- [Initialization](#initialization)
- [Usage of the llfs API](#usage-of-the-llfs-api)
- [The `llfs_file_t` struct](#the-llfs_file_t-struct)
- [Getting a list of files](#getting-a-list-of-files)
- [Iterator function (not recommended)](#iterator-function-not-recommended)
- [Reading a file](#reading-a-file)
- [Getting the number of files](#getting-the-number-of-files)
- [Using the POSIX file functions](#using-the-posix-file-functions)
## Initialization
Before using the llfs API, or the file related POSIX (fopen, fgetc, ...) functions, the filesystem must be initialized by calling `llfs_init()`.
## Usage of the llfs API
### The `llfs_file_t` struct
The `llfs_file_t` struct contains information about a file in the filesystem.
```c
typedef struct {
const uint8_t* data; // Pointer to the file data (len bytes)
const char* name; // Null-terminated string with the filename
size_t len; // Length of the file data
} llfs_file_t;
```
The data pointer points to the data of the file in the filesystem, and can be used to read the file.
### Getting a list of files
```c
#include "llfs.h"
void main(void) {
llfs_init();
// Allocate space for 10 files
llfs_file_t file_list[10];
@@ -33,13 +68,50 @@ Result:
[Info] (2031) [main]: File: file1.txt, size: 77
```
It is also possible to use a file extension filter (e.g. `*.bmp`, `*.txt`, `*.py`).
```c
// ...
size_t file_count = llfs_file_list(file_list, 10, "*.bmp");
// ...
```
This will only return files with the `.bmp` extension.
````
[Info] (2009) [main]: File: image.bmp, size: 9270
[Info] (2019) [main]: File: image2.bmp, size: 7738
````
#### Iterator function (not recommended)
It is also possible to iterate through the files without allocating an array.
When the memory pointer is `NULL`, the iterator will start at the beginning of the file list.
Each call to `llfs_next_file()` will return the next file in the list,
if a filter is specified files that don't match the filter will be skipped.
```c
#include "llfs.h"
void main(void) {
llfs_init();
// Get the file list
void* mem = NULL; // Pointer for internal use by the llfs library
llfs_file_t* file;
while ((file = llfs_next_file(&mem, ".bmp")) != NULL) {
LOG_INFO(TAG, "File: %s", file->name);
}
}
```
While this method doesn't require allocating memory for the file list,
it is slower than the previous method due to the overhead calling the function for each file.
Additionally, the required memory for the filelist is very small, so it's recommended to use the first method.
### Reading a file
```c
#include "llfs.h"
void main(void) {
llfs_init();
// Get a file by name
llfs_file_t *file = llfs_file_open("filename with a space.txt");
llfs_file_t* file = llfs_file_open("filename with a space.txt");
if (file != NULL) {
// Print the file name, size and data
@@ -55,3 +127,37 @@ Result:
[Info] (2040) [main]: File found: filename with a space.txt, size: 61
[Info] (2047) [main]: File data: This is a file with a space in it's filename.
```
### Getting the number of files
```c
#include "llfs.h"
void main(void) {
llfs_init();
// Get the number of files
size_t file_count = llfs_file_count();
// Print the number of files
LOG_INFO(TAG, "File count: %d", file_count);
}
```
## Using the POSIX file functions
The llfs library also supports the POSIX file functions.
As the file system is read-only, write functions are not implemented.
There is also a limit on the number of files that can be open concurrently,
this is set by the `POSIX_MAX_FILES` macro in `llfs.c`.
The default value is 10, but there are already 3 files in use (stdin, stdout, stderr),
so the maximum number of files that can be open is 7.
The following functions are tested and working, but other functions might also work:
- `fopen`
- `fclose`
- `fgetc`
- `fread`
- `fseek`
- `ftell`
- `rewind`
- `fstat`
- `fileno`

33
project/Core/Inc/llfs.h
View File

@@ -2,6 +2,7 @@
* @file llfs.h
* @brief Linked List Filesystem header (llfs)
* @author Lorenz C.
* @version 0.1.1
*/
#ifndef LLFS_H
@@ -30,20 +31,25 @@ struct llfs_data_file {
const struct llfs_data_file* next;
};
/**
* @brief Initialize the llfs filesystem
* @note This function should be called before any other llfs function or POSIX file operation (e.g. fopen, fread, ...)
* @return 0 on success
*/
int8_t llfs_init(void);
/**
* @brief Get a list of files in the filesystem
* Get a list of all the files in the filesystem.
*
* Use the filter to filter out files with a filename that do not match the filter. (e.g. "*.txt" or "*.png|*.jpg") (not
* implemented yet) Multiple filters can be used by separating them with a pipe (|).
* Use the filter to filter out files with a filename that do not match the file extension filter.
*
* The following members of the llfs_file_t struct are set: name, len and data. @ref llfs_file_t
*
* @todo Implement file filter
*
* @param[out] file_list A pointer to an array of llfs_file_t to store the files in @ref llfs_file_t
* @param[in] max_files The maximum number of files to return (size of file_list)
* @param[in] filter A string with file extensions to filter out. (e.g. "*.txt" or "*.png|*.jpg")
* @param[in] filter A string with the file extensions to filter out. (e.g. "*.txt" or "*.png")
* @return The number of files returned
*/
size_t llfs_file_list(llfs_file_t* file_list, size_t max_files, char* filter);
@@ -58,4 +64,23 @@ size_t llfs_file_list(llfs_file_t* file_list, size_t max_files, char* filter);
*/
llfs_file_t* llfs_file_open(const char* name);
/**
* @brief Iterate over all files in the filesystem
* For each call (with the same mem pointer) the next file in the filesystem is returned.
* The first call should be with mem = NULL.
* If a filter is specified, only files with a filename that matches the filter are returned.
*
* @param[in, out] mem A pointer to a void* that is used internally to keep track of the current file
* @param[in] filter A string with file extension to filter out. (e.g. "*.txt" or "*.png")
* @return The next file in the filesystem or NULL if there are no more files @ref llfs_file_t
*/
llfs_file_t* llfs_next_file(void** mem, char* filter);
/**
* @brief Get the number of files in the filesystem
*
* @return The number of files in the filesystem
*/
size_t llfs_file_count(void);
#endif // LLFS_H

408
project/Core/Src/llfs.c
View File

@@ -2,34 +2,82 @@
* @file llfs.c
* @brief Linked List Filesystem implementation (llfs)
* @author Lorenz C.
* @todo Implement file extension filter
* @version 0.1.1
*/
#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>
#define LOGGER_LEVEL_WARN
#include "llfs.h"
#include "log.h"
/**
* @brief The maximum number of files that can be opened concurrently using the POSIX API
*/
#define POSIX_MAX_FILES 10
extern struct llfs_data_file* llfs_root;
const char* TAG = "llfs";
size_t file_count = 0;
FILE* file_table[POSIX_MAX_FILES];
static int new_file_table_entry(void);
static int free_file_table_entry(int file_id);
static FILE* file_id_to_stream(int file_id);
static uint8_t file_ext_cmp(const char* filename, const char* ext);
int8_t llfs_init(void) {
LOG_DEBUG(TAG, "Initializing llfs");
// Initialize the file table
for (int i = 0; i < POSIX_MAX_FILES; i++) {
file_table[i] = NULL;
}
// Add stdin, stdout and stderr to the file table
file_table[STDIN_FILENO] = stdin;
file_table[STDOUT_FILENO] = stdout;
file_table[STDERR_FILENO] = stderr;
return 0;
}
// TODO: Implement file extension filter
size_t llfs_file_list(llfs_file_t* file_list, size_t max_files, char* filter) {
size_t file_count = 0;
const struct llfs_data_file* file = llfs_root;
size_t count = 0; // Number of files found
const struct llfs_data_file* file = llfs_root; // Pointer to the current file
LOG_DEBUG(TAG, "Getting file list with filter: %s", filter);
if (filter != NULL) {
LOG_DEBUG(TAG, "Filtering files with filter: %s", filter);
if (filter[0] == '*') {
filter++;
}
}
// Iterate over all files in the filesystem
while (file != NULL && file_count < max_files) {
file_list[file_count].data = file->data;
file_list[file_count].name = file->name;
file_list[file_count].len = file->len;
file_count++;
// Filter out files with a filename that does not match the filter
if (filter != NULL) {
if (!file_ext_cmp(file->name, filter)) {
file = file->next;
continue;
}
}
// Add the file to the file list
file_list[count].data = file->data;
file_list[count].name = file->name;
file_list[count].len = file->len;
// Move to the next file
count++;
file = file->next;
}
LOG_DEBUG(TAG, "Files found: %d", file_count);
return file_count;
LOG_DEBUG(TAG, "Files found: %d", count);
return count;
}
llfs_file_t* llfs_file_open(const char* name) {
@@ -48,3 +96,341 @@ llfs_file_t* llfs_file_open(const char* name) {
LOG_DEBUG(TAG, "File not found: %s", name);
return NULL;
}
llfs_file_t* llfs_next_file(void** mem, char* filter) {
struct llfs_data_file* prev_file = (struct llfs_data_file*)*mem;
uint8_t filter_ok = 0;
if (prev_file == NULL) {
prev_file = llfs_root;
} else {
prev_file = (struct llfs_data_file*)prev_file->next;
}
// If a filter is specified, only return files that match the filter
if (filter != NULL) {
LOG_DEBUG(TAG, "Filtering files with filter: %s", filter);
// Remove the '*' from the filter
if (filter[0] == '*') {
filter++;
}
while (prev_file != NULL) {
if (file_ext_cmp(prev_file->name, filter)) {
filter_ok = 1;
break;
}
prev_file = (struct llfs_data_file*)prev_file->next;
}
}
*mem = (void*)prev_file;
return (llfs_file_t*)prev_file;
}
size_t llfs_file_count(void) {
if (file_count == 0) {
const struct llfs_data_file* file = llfs_root;
while (file != NULL) {
file_count++;
file = file->next;
}
}
return file_count;
}
/**
* @brief Newlib open implementation
*
* @param path
* @param flags
* @param mode
* @return
*/
int _open(char* path, int flags, int mode) {
int file_id;
FILE* stream;
errno = 0;
llfs_file_t* llfs_file;
// Add a new entry to the file table
file_id = new_file_table_entry();
if (file_id < 0) {
LOG_WARN(TAG, "Failed to add new file table entry. %s", strerror(errno));
return -1;
}
// Get the stream associated with the file id
stream = file_id_to_stream(file_id);
if (stream == NULL) {
LOG_WARN(TAG, "Failed to get file table entry. %s", strerror(errno));
free_file_table_entry(file_id);
return -1;
}
// Get the file from the llfs filesystem
llfs_file = llfs_file_open(path);
if (llfs_file == NULL) {
LOG_DEBUG(TAG, "Failed to open file: %s", path);
free_file_table_entry(file_id);
return -1;
}
// Initialize the stream
stream->_cookie = (void*)llfs_file;
stream->_offset = 0;
stream->_flags = __SRD;
return file_id;
}
/**
* @brief Newlib close implementation
*
* @param file_id
* @return
*/
int _close(int file_id) {
FILE* stream;
// Get the stream associated with the file id
stream = file_id_to_stream(file_id);
if (stream == NULL) {
LOG_WARN(TAG, "Failed to get file table entry. %s", strerror(errno));
return -1;
}
// Remove the entry from the file table
if (free_file_table_entry(file_id) < 0) {
LOG_WARN(TAG, "Failed to remove file table entry. %s", strerror(errno));
return -1;
}
return 0;
}
/**
* @brief Newlib read implementation
*
* @param file_id
* @param ptr
* @param len
* @return
*/
int _read(int file_id, char* ptr, int len) {
FILE* stream;
llfs_file_t* llfs_file;
size_t bytes_read;
// Read from stdin, stdout and stderr is not supported
if (file_id == STDIN_FILENO || file_id == STDOUT_FILENO || file_id == STDERR_FILENO) {
LOG_DEBUG(TAG, "Trying to read from stdin, stdout or stderr: %d", file_id);
return -1;
}
// Get the stream associated with the file id
stream = file_id_to_stream(file_id);
if (stream == NULL) {
LOG_WARN(TAG, "Failed to get file table entry. %s", strerror(errno));
return -1;
}
// Get the file from the llfs filesystem associated with the stream
llfs_file = (llfs_file_t*)stream->_cookie;
if (llfs_file == NULL) {
LOG_WARN(TAG, "Failed to get llfs file associated with stream: %d", file_id);
return -1;
}
// Calculate the number of bytes to read (limited by the file size)
bytes_read = llfs_file->len - stream->_offset;
if (bytes_read > len) {
bytes_read = len;
} else if (bytes_read == 0) { // End of file
stream->_flags |= __SEOF;
return 0;
}
// Copy the data over to the dst buffer
memcpy(ptr, llfs_file->data + stream->_offset, bytes_read);
stream->_offset += (off_t)bytes_read;
return (int)bytes_read;
}
/**
* @brief Newlib isatty implementation
*
* @param file
* @return 1 if the file is stdin, stdout or stderr, 0 otherwise
*/
int isatty(int file) {
if (file == STDIN_FILENO || file == STDOUT_FILENO || file == STDERR_FILENO) {
return 1;
}
return 0;
}
/**
* @brief Newlib lseek implementation
*
* @param file
* @param ptr
* @param dir
* @return
*/
int _lseek(int file, int ptr, int dir) {
FILE* stream;
if (file == STDIN_FILENO || file == STDOUT_FILENO || file == STDERR_FILENO) {
LOG_DEBUG(TAG, "Trying to seek stdin, stdout or stderr: %d", file);
return -1;
}
stream = file_id_to_stream(file);
if (stream == NULL) {
LOG_WARN(TAG, "Failed to get file table entry. %s", strerror(errno));
return -1;
}
switch (dir) {
case SEEK_SET:
stream->_offset = ptr;
break;
case SEEK_CUR:
stream->_offset += ptr;
break;
case SEEK_END:
stream->_offset = (off_t)((llfs_file_t*)stream->_cookie)->len + ptr;
break;
default:
LOG_WARN(TAG, "Invalid seek direction: %d", dir);
return -1;
}
return 0;
}
/**
* @brief Newlib fstat implementation
*
* @param[in] file
* @param[out] st
* @return
*/
int _fstat(int file, struct stat* st) {
FILE* stream;
llfs_file_t *llfs_file;
// Check if the file is stdin, stdout or stderr
if (file == STDIN_FILENO || file == STDOUT_FILENO || file == STDERR_FILENO) {
st->st_mode = S_IFCHR; // Character special file
return 0;
}
// Get the stream associated with the file id
stream = file_id_to_stream(file);
if (stream == NULL) {
LOG_WARN(TAG, "Failed to get file table entry. %s", strerror(errno));
return -1;
}
// Get the file from the llfs filesystem associated with the stream
llfs_file = (llfs_file_t*)stream->_cookie;
if (llfs_file == NULL) {
LOG_WARN(TAG, "Failed to get llfs file associated with stream: %d", file);
return -1;
}
st->st_mode = S_IFREG; // Regular file
st->st_size = (off_t)llfs_file->len;
return 0;
}
/**
* @brief Create a new entry in the file table
*
* @return The file id or -1 if an error occurred. See errno for more information.
*/
static int new_file_table_entry(void) {
FILE* stream;
// Try to find an empty entry in the file table
for (int i = 0; i < POSIX_MAX_FILES; i++) {
if (file_table[i] == NULL) {
stream = (FILE*)malloc(sizeof(FILE));
if (stream == NULL) {
errno = ENOMEM; // Out of memory
return -1;
}
file_table[i] = stream;
return i;
}
}
// No empty entry found
errno = ENFILE; // Too many open files
return -1;
}
/**
* @brief Remove an entry from the file table
*
* @param[in] file_id The file id to remove
* @return 0 if successful, -1 if an error occurred. See errno for more information.
*/
static int free_file_table_entry(int file_id) {
// Check if the file id is valid
if (file_id < 0 || file_id >= POSIX_MAX_FILES) {
errno = EBADF; // Bad file number
return -1;
}
// Remove the entry from the file table
free(file_table[file_id]);
file_table[file_id] = NULL;
return 0;
}
/**
* @brief Get the stream associated with a file id
*
* @param[in] file_id The file id to get the stream for
* @return The stream or NULL if an error occurred. See errno for more information.
*/
static FILE* file_id_to_stream(int file_id) {
if (file_id < 0 || file_id >= POSIX_MAX_FILES) {
errno = EBADF; // Bad file number
return NULL;
}
return file_table[file_id];
}
/**
* @brief Check if a filename ends with a given extension
*
* @param[in] filename The filename to check
* @param[in] ext The extension to check for
* @return 1 if the filename ends with the extension, 0 otherwise
*/
static uint8_t file_ext_cmp(const char* const filename, const char* const ext) {
uint8_t ext_len = strlen(ext);
uint8_t filename_len = strlen(filename);
if (filename_len < ext_len) {
return 0;
}
// Compare backwards
for (uint8_t i = 0; i < ext_len; i++) {
if (filename[filename_len - i] != ext[ext_len - i]) {
return 0;
}
}
return 1;
}

1163
project/Core/Src/main.c
View File

File diff suppressed because it is too large Load Diff

10
project/Core/Src/syscalls.c
View File

@@ -89,14 +89,14 @@ __attribute__((weak)) int _write(int file, char *ptr, int len)
return len;
}
int _close(int file)
__attribute__((weak)) int _close(int file)
{
(void)file;
return -1;
}
int _fstat(int file, struct stat *st)
__attribute__((weak)) int _fstat(int file, struct stat *st)
{
(void)file;
st->st_mode = S_IFCHR;
@@ -109,7 +109,7 @@ int _isatty(int file)
return 1;
}
int _lseek(int file, int ptr, int dir)
__attribute__((weak)) int _lseek(int file, int ptr, int dir)
{
(void)file;
(void)ptr;
@@ -117,7 +117,7 @@ int _lseek(int file, int ptr, int dir)
return 0;
}
int _open(char *path, int flags, ...)
__attribute__((weak)) int _open(char *path, int flags, ...)
{
(void)path;
(void)flags;
@@ -145,7 +145,7 @@ int _times(struct tms *buf)
return -1;
}
int _stat(char *file, struct stat *st)
__attribute__((weak)) int _stat(char *file, struct stat *st)
{
(void)file;
st->st_mode = S_IFCHR;