X-Git-Url: http://git.silcnet.org/gitweb/?p=runtime.git;a=blobdiff_plain;f=TODO;h=e7693efc7bf6f951fa6c0fa8b2c023527a5e1a94;hp=6a1e9923b485d46aa6d76645886bec7726cfb1ae;hb=HEAD;hpb=84f9b303d7df354d83ca14a56fedafd67f3116b8 diff --git a/TODO b/TODO index 6a1e9923..e7693efc 100644 --- a/TODO +++ b/TODO @@ -1,541 +1,161 @@ -TODO for 1.2 And Beyond -======================= +SILC Runtime Toolkit 1.2 and beyond +=================================== NOTE: Any item that doesn't have (***DONE) in it, isn't done yet. The (***TESTING NEEDED) means that the item has been done but not yet properly tested. -NOTE: A TODO entry does not mean that it is ever going to be done. Some -of the entries may be just ideas, good, bad or ugly. If you want to work -on some of the TODO entries simply let us know about it by dropping a note +NOTE: A TODO entry does not mean that it is ever going to be done. Some +of the entries may be just ideas, good, bad or ugly. If you want to work +on some of the TODO entries simply let us know about it by dropping a note to silc-devel mailing list or appear on 'silc' channel on SILCNet. -General -======= - - o Create apps/tutorial containing various Toolkit API tutorials. - - o The Toolkit split. The Toolkit is to be splitted in parts. How many - parts and what the parts are isn't decided yet. Each part is a separate - software package. Current thinking is of the following: - - SILC Toolkit SILC protocol, client and server library - SILC Runtime Toolkit runtime library - SILC Crypto Toolkit crypto, asn1, math, skr, pgp, etc. - - The rationale for this is of course that other than SILC projects - might like to use the various libraries SILC Toolkit provides, but - naturally they don't want the bloat of SILC protocol related stuff. - - The Runtime library in SILC Toolkit is a general purpose runtime library, - like Glib and APR are. The runtime library is to be developed further - to provide alternative to Glib and APR. - - The Crypto library in SILC Toolkit is a general purpose crypto library - providing pretty nice APIs compared to many other crypto libraries, - especially OpenSSL. The Crypto library is to be developed further - to include support for OpenPGP, X.509 and SSH2. - - -lib/silccore -============ - - o SILC_PACKET_FLAG_ACK support. Implement ACK packet and packet payload - to silcpacket.c. - - o All payload encoding routines should take SilcStack as argument. - - o All payload test routines into lib/silccore/tests/. - - -lib/silcclient, The Client Library -================================== - - o peer-to-peer private messages - - o Private message key request notification to application. See XXX in - client_prvmsg.c. - - o in JOIN notify handle resolving that timedout. Currently the user is - never joined the channel if this happens. What to do if message is - received from user that hasn't been resolved/joined? - - o Message ACKing support. - - o in /cmode and /cumode with +r, maybe the public key and private key - could be just some "string", which would then match to "string.pub" and - "string.prv". - - Runtime library, lib/silcutil/ ============================== - o Fix universal time decoding (doesn't accept all formats) in silctime.c. + o Add SILC Zip API, compression. - o Add functions to manipulate environment variables. + o file removing, chmod, rmmod, etc. chdir, rmdir, stat, etc. to + lib/silcutil/silcfileutil.h. - SilcBool silc_setenv(const char *variable, const char *value); - const char *silc_getenv(const char *variable); - SilcBool silc_clearenv(const char *variable); + o mmap API - o Add functions to loading shared/dynamic object symbols (replaces the - SIM library (lib/silcsim) and introduces generic library). + o Multicast support. - SilcDll silc_dll_load(const char *object_path, SilcDllFlags flags); - void silc_dll_close(SilcDll dll); - void *silc_dll_getsym(SilcDll dll, const char *symbol); - const char *silc_dll_error(SilcDll dll); + o Implement silc_get_num_cpu. - o Add directory opening/traversing functions + o silc_file_readfile and silc_file_writefile should perhaps do the thing + with mmap, as it's probably a bit faster. - o silc_getopt routines + o silc_malloc et. al. to respect --with-alignment. - o silc_hash_table_replace -> silc_hash_table_set. Retain support for - silc_hash_table_replace as macro. + o Fix universal time decoding (doesn't accept all formats) in silctime.c. - o The SILC Event signals. Asynchronous events that can be created, - connected to and signalled. Either own event routines or glued into - SilcSchedule: - - SilcTask silc_schedule_task_add_event(SilcSchedule schedule, - const char *event, ...); - SilcBool silc_schedule_event_connect(SilcSchedule schedule, - const char *event, - SilcTaskCallback event_callback, - void *context); - SilcBool silc_schedule_event_signal(SilcSchedule schedule, - const char *event, ...); - - Example: - silc_schedule_task_add_event(schedule, "connected", - SILC_PARAM_UI32_INT, - SILC_PARAM_BUFFER, - SILC_PARAM_END); - silc_schedule_event_connect(schedule, "connected", connected_cb, ctx); - silc_schedule_event_signal(schedule, "connected", integer, buf, - SILC_PARAM_END); - SILC_TASK_CALLBACK(connected_cb) - { - FooCtx ctx = context; - va_list args; - SilcUInt32 integer; - SilcBuffer buf; - - va_start(args, context); - integer = va_arg(args, SilcUInt32); - buf = va_arg(args, SilcBuffer); - va_end(args); - ... - } - - Problems: Events would be SilcSchedule specific, and would not work on - multi-thread/multi-scheduler system. The events should be copyable - between schedulers. - - o Structured log messages to Log API. Allows machine readable log - messages. Would allow sending of any kind of data in a log message. - - o Base64 to an own API - - o Timer API + o Additional scheduler changes: optimize silc_schedule_wakeup. Wakeup + only if the scheduler is actually waiting something. If it is + delivering tasks wakeup is not needed. + + o silc_stringprep to non-allocating version. o Add builtin SOCKS and HTTP Proxy support, well the SOCKS at least. SILC currently supports SOCKS4 and SOCKS5 but it needs to be compiled in separately. - o silc_stringprep to non-allocating version. - - o SilcStack aware SilcHashTable. - - o SilcStack aware SilcDList. - - o Thread pool API. - - typedef void (*SilcThreadPoolFunc)(SilcSchedule schedule, - void *context); - - /* Allocate thread pool with at least `min_threads' and at most - `max_threads' many threads. If `stack' is non-NULL all memory - is allocated from the `stack'. If `start_min_threads' is TRUE - this will start `min_threads' many threads immediately. */ - SilcThreadPool silc_thread_pool_alloc(SilcStack stack, - SilcUInt32 min_threads, - SilcUInt32 max_threads, - SilcBool start_min_threads); - - /* Free thread pool. If `wait_unfinished' is TRUE this will block - and waits that all remaining active threads finish before freeing - the pool. */ - void silc_thread_pool_free(SilcThreadPool tp, SilcBool wait_unfinished); - - /* Run `run' function with `run_context' in one of the threads in the - thread pool. Returns FALSE if the thread pool is being freed. If - there are no free threads left in the pool this will queue the - the `run' and will call it once a thread becomes free. - - If `completion' is non-NULL it will be called to indicate completion - of the `run' function. If `schedule' is non-NULL the `completion' - will be called through the scheduler in the main thread. If it is - NULL the `completion' is called directly from the thread after the - `run' has returned. */ - SilcBool silc_thread_pool_run(SilcThreadPool tp, - SilcSchedule schedule, - SilcThreadPoolFunc run, - void *run_context, - SilcThreadPoolFunc completion, - void *completion_context); - - /* Modify the amount of maximum threads of the pool. */ - void silc_thread_pool_set_max_threads(SilcThreadPool tp, - SilcUInt32 max_threads); - - /* Returns the amount of maximum size the pool can grow. */ - SilcUInt32 silc_thread_pool_get_max_threads(SilcThreadPool tp); - - /* Returns the amount of free threads in the pool currently. */ - SilcUInt32 silc_thread_pool_get_free_threads(SilcThreadPool tp); - - /* Stops all free and started threads. The minumum amount of threads - specified to silc_thread_pool_alloc always remains. */ - void silc_thread_pool_stop_free_threads(SilcThreadPool tp); - - o Compression routines are missing. The protocol supports packet - compression thus it must be implemented. SILC Zip API must be - defined. - - (o Generic SilcStatus or SilcResult that includes all possible status and - error conditions, including those of SILC protocol. Though, the SILC - protocol related status (currently in silcstatus.h) cannot be in - runtime library) maybe - - (o SILC specific socket creation/closing routines to silcnet.h, wrappers - to all send(), recv(), sendto() etc. Bad thing is that we'd have to - define all socket options, sockaddrs, etc.) maybe - - (o mmap) maybe - - -lib/silcutil/symbian/ -===================== - - o Something needs to be thought to the logging globals as well, - like silc_debug etc. They won't work on EPOC. Perhaps logging - and debugging is to be disabled on EPOC. The logging currently works - by it cannot be controlled, same with debugging. - + o Add silc_xml_parse_stream to parse SilcStream XML stream. -SFTP Library, lib/silcsftp/ -=========================== + o SILC XML API (wrapper to expat). (***DONE) - o Read prefetch (read-ahead, reading ahead of time). Maybe if this can - be done easily. + o Bring silchttp HTTP server library to SRT. (***DONE) + o Simple SILC Rand API for pseudo-random numbers. (***DONE) -SKR Library, lib/silcskr/ -========================= - - o Add fingerprint as search constraint. - - o Add OpenPGP support. Adding, removing, fetching PGP keys. (Keyring - support?) - - o Add support for importing public keys from a directory and/or from a - file. Add support for exporting the repository (different formats for - different key types?). - - o Change the entire silc_skr_find API. Remove SilcSKRFind and just simply - add the find constraints as variable argument list to silc_skr_find, eg: - - silc_skr_find(skr, schedule, callback, context, - SILC_SKR_FIND_PUBLIC_KEY, public_key, - SILC_SKR_FIND_COUNTRY, "FI", - SILC_SKR_FIND_USAGE, SILC_SKR_USAGE_AUTH, - SILC_SKR_FIND_END); - - NULL argument would be ignored and skipped. - - o Add OR logical rule in addition of the current default AND, eg: - - // Found key(s) MUST have this public key AND this country. - silc_skr_find(skr, schedule, callback, context, - SILC_SKR_FIND_RULE_AND, - SILC_SKR_FIND_PUBLIC_KEY, public_key, - SILC_SKR_FIND_COUNTRY, "FI", - SILC_SKR_FIND_END); - - // Found key(s) MUST have this public key OR this key context - silc_skr_find(skr, schedule, callback, context, - SILC_SKR_FIND_RULE_OR, - SILC_SKR_FIND_PUBLIC_KEY, public_key, - SILC_SKR_FIND_CONTEXT, key_context, - SILC_SKR_FIND_END); - - -Crypto Library, lib/silccrypt/ -============================== - - o Add fingerprint to SilcSILCPublicKey and retrieval to silcpk.h, and - possibly to silcpkcs.h. + o Add directory opening/traversing functions (***DONE, TODO Windows & Symbian) - /* Return fingerprint of the `public_key'. Returns also the algorithm - that has been used to make the fingerprint. */ - const unsigned char * - silc_pkcs_get_fingerprint(SilcPublicKey public_key, - const char **hash_algorithm, - SilcUInt32 *fingerprint_len); + o regex from /lib/contrib to lib/silcutil, define SILC Regex API. (***DONE) - o Change SILC PKCS API to asynchronous, so that accelerators can be used. - All PKCS routines should now take callbacks as argument and they should - be delivered to SilcPKCSObject and SilcPKCSAlgorithm too. + o Add functions to manipulate environment variables. (***DONE) - /* Signature computation callback */ - typedef void (*SilcPKCSSignCb)(SilcBool success, - const unsigned char *signature, - SilcUInt32 signature_len, - void *context); - - /* Signature verification callback */ - typedef void (*SilcPKCSVerifyCb)(SilcBool success, void *context); - - /* Encryption callback */ - typedef void (*SilcPKCSEncryptCb)(SilcBool success, - const unsigned char *encrypted, - SilcUInt32 encrypted_len, - void *context); - - /* Decryption callback */ - typedef void (*SilcPKCSDecryptCb)(SilcBool success, - const unsigned char *decrypted, - SilcUInt32 decrypted_len, - void *context); - - Either add new _async functions or add the callbacks to existing API - and if the callback is NULL then the API is not async and if provided - it may be async. For example; - - SilcBool silc_pkcs_sign(SilcPrivateKey private_key, - unsigned char *src, SilcUInt32 src_len, - unsigned char *dst, SilcUInt32 dst_size, - SilcUInt32 *dst_len, - SilcBool compute_hash, SilcHash hash, - SilcPKCSSignCb async_sign, - void *async_sign_context); - - (if this is done then there's no reason why the buffers in the - callbacks cannot be the ones user gives here) or allow only async: - - SilcBool silc_pkcs_sign(SilcPrivateKey private_key, - unsigned char *src, SilcUInt32 src_len, - SilcBool compute_hash, SilcHash hash, - SilcPKCSSignCb async_sign, - void *async_sign_context); - - or add new: - - SilcBool silc_pkcs_sign_async(SilcPrivateKey private_key, - unsigned char *src, SilcUInt32 src_len, - SilcBool compute_hash, SilcHash hash, - SilcPKCSSignCb async_sign, - void *async_sign_context); - - o Change PKCS Algorithm API to take SilcPKCSAlgorithm as argument to - encrypt, decrypt, sign and verify functions. We may need to for exmaple - check the alg->hash, supported hash functions. Maybe deliver it also - to all other functions in SilcPKCSAlgorithm to be consistent. - - o Add DSS support. Take implementation from Tom or make it yourself. - - o Implement the defined SilcDH API. The definition is in - lib/silccrypt/silcdh.h. - - o All cipher, hash, hmac etc. allocation routines should take their name - in as const char * not const unsigned char *. - - o ECDSA and ECDH - - -SILC Accelerator Library -======================== - - o SILC Accelerator API. Provides generic way to use different kind of - accelerators. Basically implements SILC PKCS API so that SilcPublicKey - and SilcPrivateKey can be used but they call the accelerators. - - Something in the lines of (preliminary): - - /* Register accelerator to system */ - SilcBool silc_acc_register(const SilcAccelerator acc); - - /* Unregister accelerator */ - SilcBool silc_acc_unregister(const SilcAccelerator acc); - - /* Find existing accelerator. `name' is accelerators name and - `params' is optional accelerator specific parameters. */ - SilcAccelerator silc_acc_find(const char *name, const char *params); - - /* Return accelerator's displayable name */ - const char *silc_ac_get_display_name(SilcAccelerator acc); - - /* Accelerate `public_key'. Return accelerated public key. */ - SilcPublicKey silc_acc_public_key(SilcAccelerator acc, - SilcPublicKey public_key); - - /* Accelerate `private_key'. Returns accelerated private key. */ - SilcPrivateKey silc_acc_private_key(SilcAccelerator acc, - SilcPrivateKey private_key); - - /* Return the underlaying public key */ - SilcPublicKey silc_acc_get_public_key(SilcAccelerator acc, - SilcPublicKey public_key); - - /* Return the underlaying private key */ - SilcPrivateKey silc_acc_get_private_key(SilcAccelerator acc, - SilcPrivateKey private_key); - - typedef struct SilcAcceleratorObject { - const char *name; /* Accelerator's name */ - const char *display_name; /* Displayable name */ - SilcAcceleratorType type; /* Accelerator type */ - union { - struct { - SilcPKCSObject *pkcs; /* PKCS, may be NULL */ - SilcPKCSAlgorithm *algorithm; /* Accelerator */ - } pkcs; - - struct { - - } cipher; - } u; - } *SilcAccelerator, SilcAcceleratorStruct; - - SilcPublicKey->SilcSILCPublicKey->RsaPublicKey accelerated as: - SilcPublicKey->SilcSILCPublicKey->SilcAcceleratorSoftware->RsaPublicKey or - SilcPublicKey->SilcSILCPublicKey->SilcAcceleratorPublicKey-> - SilcAcceleratorSoftware->RsaPublicKey - - The former one if u.pkcs.pkcs == NULL. - - o Implement software accelerator. It is a thread pool system where the - public key and private key operations are executed in threads. - - This implements SilcPKCSAlgorithm (and SilcPKCSObject if needed) that - implements the thread acclerated system. - - (o Symmetric key cryptosystem acceleration? They are always sycnhronouos - even with hardware acceleration so the crypto API shouldn't require - changes.) maybe - - -lib/silcmath -============ - - o Import TFM. Talk to Tom to add the missing functions. Use TFM in - client and client library, but TMA in server, due to the significantly - increased memory consumption with TFM, and the rare need for public - key operations in server. - - We want TFM's speed but not TFM's memory requirements. Talk to Tom - about making the TFM mp dynamic just as it is in LTM. - - o The SILC MP API function must start returning indication of success - and failure of the operation. - - o Do SilcStack support for silc_mp_init, silc_mp_init_size and other - any other MP function (including utility ones) that may allocate - memory. - - o All utility functions should be made non-allocating ones. - - -SILC XML Library, lib/silcxml/ -============================== + o Add functions to loading shared/dynamic object symbols (replaces the + SIM library (lib/silcsim) and introduces generic library). Add this + to lib/silcutil/silcdll.[ch]. (***TESTING NEEDED WIN32, TODO Symbian) - o SILC XML API (wrapper to expat). The SILC XML API should follow and - resemble Simple API for XML (SAX). + o silc_getopt routines (***DONE) + o The SILC Event signals. Asynchronous events that can be created, + connected to and signalled. Either own event routines or glued into + SilcSchedule. (***DONE) -lib/silcske/silcske.[ch] -======================== + o If the event signals are added, the SILC_PARAM_* stuff needs to be + moved from silcbuffmt.h to silctypes.h or something similar. (***DONE) - o Ratelimit to UDP/IP transport for incoming packets. + o In case the SILC Events are done we shall create a new concept of + parent and child SilcSchedule's. When new SilcSchedule is created a + parent can be associated to it. This association could be done either + directly by the parent or by any other children. This way the signals + would in effect be global and would reach all children schedulers. + This relationship would be associative only. The schedulers are still + independent and run independently from each other. All schedulers + would be linked and could be accessed from any of the schedulers. + It should be possible to retrieve the parent and enumerate all children + from any of the schedulers. + (***DONE) -lib/silcasn1 -============ + o Base64 to an own API (***DONE) - o Negative integer encoding is missing, add it. + o Timer API (***DONE) - o SILC_ASN1_CHOICE should perhaps return an index what choice in the - choice list was found. Currently it is left for caller to figure out - which choice was found. + o silc_hash_table_replace -> silc_hash_table_set. Retain support for + silc_hash_table_replace as macro. (***DONE) - o SILC_ASN1_NULL in decoding should return SilcBool whether or not - the NULL was present. It's important when it's SILC_ASN1_OPTIONAL - and we need to know whether it was present or not. + o SilcStack aware SilcHashTable. (***DONE) + o SilcStack aware SilcDList. (***DONE) -lib/silcpgp -=========== + o Thread pool API. Add this to lib/silcutil/silcthread.[ch]. (***DONE) - o OpenPGP certificate support, allowing the use of PGP public keys - in SILC. + o Add new functions to SilcStack API in lib/silcutil/silcstack.[ch]. Add + silc_stack_[set|get]_alignment. It defines the default alignment used + when allocating memory from stack. It can be used to specify special + alignments too when needed (such as for hardware devices like crypto + accelerators). Move also the low level silc_stack_malloc and + silc_stack_realloc from silcstack_i.h to silcstack.h. Remove the + _ua unaligned memory allocation routines. Remove unaligned memory + allocation possibility. (***DONE) + o silc_stack_alloc shouldn't require multiple by 8 size argument, it + should figure it out itself. (***DONE) -lib/silcssh -=========== + o Add '%@' format to silc_snprintf functions. + (***DONE) - o SSH2 public key/private key support, allowing the use of SSH2 keys - in SILC. RFC 4716. + o SILC Tls (Thread-local storage) API to lib/silcutil/silcthread.[ch]. + (***DONE) + o Change silc_gettimeofday on Unix to use clock_gettime with REALTIME + clock if it is available, otherwise use gettimeofday(). (***DONE) -lib/silcpkix -============ + o Generic SilcResult that includes all possible status and + error conditions and generic errno API. (***DONE) - o PKIX implementation + (o Structured log messages to Log API. Allows machine readable log + messages. Would allow sending of any kind of data in a log message.) maybe + (o Change some stream routines (like socket stream API) to accept ANY + stream and use silc_stream_get_root to get the socket stream from the + given stream. This will make various stream APIs more easier to use + when user doesn't have to dig up the correct stream. -lib/silcserver -============== + Add silc_stream_get_root and add get_root stream operation. It + returns the root of the stream or NULL if stream doesn't have root.) maybe - o (Re)write commands/command replys. + (o SilcIpAddr abstraction. Ipv4 and Ipv6 support to the abstaction.) + maybe - o (Re)write notify handling. + (o SILC specific socket creation/closing routines to silcnet.h, wrappers + to all send(), recv(), sendto() etc. Bad thing is that we'd have to + define all socket options, sockaddrs, etc.) maybe - o The SERVER_SIGNOFF notify handing is not optimal, because it'll - cause sending of multiple SIGNOFF notify's instead of the one - SERVER_SIGNOFF notify that the server received. This should be - optimized so that the only SERVER_SIGNOFF is sent and not - SIGNOFF of notify at all (using SIGNOFF takes the idea about - SERVER_SIGNOFF away entirely). + (o Fast mutex implementation. Fast rwlock implementation. Mutex and + rwlock implementation using atomic operations.) not for now. - o Another SERVER_SIGNOFF opt/bugfix: Currently the signoff is - sent to a client if it is on same channel as the client that - signoffed. However, the entire SERVER_SIGNOFF list is sent to - the client, ie. it may receive clients that was not on the - same channel. This is actually against the specs. It must be - done per channel. It shouldn't receive the whole list just - because one client happened to be on same channel. - o Add reference counters to all Silc*Entry structures +Windows Support +=============== - o SERVICEs support (plugin, SIM) - o If client's public key is saved in the server (and doing public key - authentication) then the hostname and the username information could - be taken from the public key. Should be a configuration option! +Symbian OS Support +================== - o Add a timeout to handling incoming JOIN commands. It should be - enforced that JOIN command is executed only once in a second or two - seconds. Now it is possible to accept n incoming JOIN commands - and process them without any timeouts. THis must be employed because - each JOIN command will create and distribute the new channel key - to everybody on the channel. + o Something needs to be thought to the logging globals as well, + like silc_debug etc. They won't work on EPOC. Perhaps logging + and debugging is to be disabled on EPOC. The logging currently works + by it cannot be controlled, same with debugging. SILC Global API + MUST be used with all globals on Symbian. - o Related to above. If multiple JOINs are received in sequence perhaps - new key should be created only once, if the JOINs are handeled at the same - time. Now we create multiple keys and never end up using them because - many JOINs are processed at the same time in sequence. Only the last - key ends up being used. + o gethostname() returns "Function not implemented". Others may return + the same. We should fix that probably to use RHostResolver and + GetHostName(). - o The CMODE cipher & hmac change problem (#101). + o silc_thread_exit should call User::Exit().