-TODO for 1.2 And Beyond
-=======================
+SILC Runtime Toolkit 1.2 and beyond
+===================================
NOTE: Any item that doesn't have (***DONE) in it, isn't done yet. The
(***TESTING NEEDED) means that the item has been done but not yet properly
tested.
-NOTE: A TODO entry does not mean that it is ever going to be done. Some
-of the entries may be just ideas, good, bad or ugly. If you want to work
-on some of the TODO entries simply let us know about it by dropping a note
+NOTE: A TODO entry does not mean that it is ever going to be done. Some
+of the entries may be just ideas, good, bad or ugly. If you want to work
+on some of the TODO entries simply let us know about it by dropping a note
to silc-devel mailing list or appear on 'silc' channel on SILCNet.
to include support for OpenPGP, X.509 and SSH2.
-lib/silccore
-============
-
- o SILC_PACKET_FLAG_ACK support. Implement ACK packet and packet payload
- to silcpacket.c.
+Runtime library, lib/silcutil/
+==============================
- o All payload encoding routines should take SilcStack as argument.
+ o Unix socket support to Socket Stream API (local socket stream).
- o All payload test routines into lib/silccore/tests/.
+ o Simple SILC Rand API for pseudo-random numbers. (***DONE)
+ o file removing, chmod, rmmod, etc. chdir, rmdir, stat, etc. to
+ lib/silcutil/silcfileutil.h.
-lib/silcclient, The Client Library
-==================================
+ o mmap API
- o peer-to-peer private messages
+ o Multicast support.
- o Private message key request notification to application. See XXX in
- client_prvmsg.c.
+ o silc_file_readfile and silc_file_writefile should perhaps do the thing
+ with mmap, as it's probably a bit faster.
- o in JOIN notify handle resolving that timedout. Currently the user is
- never joined the channel if this happens. What to do if message is
- received from user that hasn't been resolved/joined?
+ o silc_malloc et. al. to respect --with-alignment.
- o Message ACKing support.
+ o Fix universal time decoding (doesn't accept all formats) in silctime.c.
- o in /cmode and /cumode with +r, maybe the public key and private key
- could be just some "string", which would then match to "string.pub" and
- "string.prv".
+ o Additional scheduler changes: optimize silc_schedule_wakeup. Wakeup
+ only if the scheduler is actually waiting something. If it is
+ delivering tasks wakeup is not needed.
+ o silc_stringprep to non-allocating version.
-Runtime library, lib/silcutil/
-==============================
+ o Add builtin SOCKS and HTTP Proxy support, well the SOCKS at least.
+ SILC currently supports SOCKS4 and SOCKS5 but it needs to be compiled
+ in separately.
- o Fix universal time decoding (doesn't accept all formats) in silctime.c.
+ o Add directory opening/traversing functions (***DONE, TODO Windows & Symbian)
- o Add functions to manipulate environment variables.
+ o regex from /lib/contrib to lib/silcutil, define SILC Regex API. (***DONE)
- SilcBool silc_setenv(const char *variable, const char *value);
- const char *silc_getenv(const char *variable);
- SilcBool silc_clearenv(const char *variable);
+ o Add functions to manipulate environment variables. (***DONE)
o Add functions to loading shared/dynamic object symbols (replaces the
SIM library (lib/silcsim) and introduces generic library). Add this
- to lib/silcutil/silcdll.[ch].
-
- SilcDll silc_dll_load(const char *object_path, SilcDllFlags flags);
- void silc_dll_close(SilcDll dll);
- void *silc_dll_getsym(SilcDll dll, const char *symbol);
- const char *silc_dll_error(SilcDll dll);
+ to lib/silcutil/silcdll.[ch]. (***TESTING NEEDED WIN32, TODO Symbian)
- o Add directory opening/traversing functions
-
- o silc_getopt routines
-
- o silc_hash_table_replace -> silc_hash_table_set. Retain support for
- silc_hash_table_replace as macro.
+ o silc_getopt routines (***DONE)
o The SILC Event signals. Asynchronous events that can be created,
connected to and signalled. Either own event routines or glued into
- SilcSchedule:
-
- SilcTask silc_schedule_task_add_event(SilcSchedule schedule,
- const char *event, ...);
- SilcBool silc_schedule_event_connect(SilcSchedule schedule,
- const char *event,
- SilcTaskCallback event_callback,
- void *context);
- SilcBool silc_schedule_event_signal(SilcSchedule schedule,
- const char *event, ...);
-
- Example:
- silc_schedule_task_add_event(schedule, "connected",
- SILC_PARAM_UI32_INT,
- SILC_PARAM_BUFFER,
- SILC_PARAM_END);
- silc_schedule_event_connect(schedule, "connected", connected_cb, ctx);
- silc_schedule_event_signal(schedule, "connected", integer, buf,
- SILC_PARAM_END);
- SILC_TASK_CALLBACK(connected_cb)
- {
- FooCtx ctx = context;
- va_list args;
- SilcUInt32 integer;
- SilcBuffer buf;
-
- va_start(args, context);
- integer = va_arg(args, SilcUInt32);
- buf = va_arg(args, SilcBuffer);
- va_end(args);
- ...
- }
-
- Problems: Events would be SilcSchedule specific, and would not work on
- multi-thread/multi-scheduler system. The events should be copyable
- between schedulers.
+ SilcSchedule. (***DONE)
o If the event signals are added, the SILC_PARAM_* stuff needs to be
- moved from silcbuffmt.h to silctypes.h or something similar.
+ moved from silcbuffmt.h to silctypes.h or something similar. (***DONE)
- o Structured log messages to Log API. Allows machine readable log
- messages. Would allow sending of any kind of data in a log message.
+ o In case the SILC Events are done we shall create a new concept of
+ parent and child SilcSchedule's. When new SilcSchedule is created a
+ parent can be associated to it. This association could be done either
+ directly by the parent or by any other children. This way the signals
+ would in effect be global and would reach all children schedulers.
- o Base64 to an own API
+ This relationship would be associative only. The schedulers are still
+ independent and run independently from each other. All schedulers
+ would be linked and could be accessed from any of the schedulers.
+ It should be possible to retrieve the parent and enumerate all children
+ from any of the schedulers.
+ (***DONE)
- o Timer API
+ o Base64 to an own API (***DONE)
- o Add builtin SOCKS and HTTP Proxy support, well the SOCKS at least.
- SILC currently supports SOCKS4 and SOCKS5 but it needs to be compiled
- in separately.
+ o Timer API (***DONE)
- o silc_stringprep to non-allocating version.
-
- o SilcStack aware SilcHashTable.
+ o silc_hash_table_replace -> silc_hash_table_set. Retain support for
+ silc_hash_table_replace as macro. (***DONE)
- o SilcStack aware SilcDList.
+ o SilcStack aware SilcHashTable. (***DONE)
- o Thread pool API. Add this to lib/silcutil/silcthread.[ch].
+ o SilcStack aware SilcDList. (***DONE)
- typedef void (*SilcThreadPoolFunc)(SilcSchedule schedule,
- void *context);
+ o Thread pool API. Add this to lib/silcutil/silcthread.[ch]. (***DONE)
- /* Allocate thread pool with at least `min_threads' and at most
- `max_threads' many threads. If `stack' is non-NULL all memory
- is allocated from the `stack'. If `start_min_threads' is TRUE
- this will start `min_threads' many threads immediately. */
- SilcThreadPool silc_thread_pool_alloc(SilcStack stack,
- SilcUInt32 min_threads,
- SilcUInt32 max_threads,
- SilcBool start_min_threads);
+ o Add new functions to SilcStack API in lib/silcutil/silcstack.[ch]. Add
+ silc_stack_[set|get]_alignment. It defines the default alignment used
+ when allocating memory from stack. It can be used to specify special
+ alignments too when needed (such as for hardware devices like crypto
+ accelerators). Move also the low level silc_stack_malloc and
+ silc_stack_realloc from silcstack_i.h to silcstack.h. Remove the
+ _ua unaligned memory allocation routines. Remove unaligned memory
+ allocation possibility. (***DONE)
- /* Free thread pool. If `wait_unfinished' is TRUE this will block
- and waits that all remaining active threads finish before freeing
- the pool. */
- void silc_thread_pool_free(SilcThreadPool tp, SilcBool wait_unfinished);
+ o silc_stack_alloc shouldn't require multiple by 8 size argument, it
+ should figure it out itself. (***DONE)
- /* Run `run' function with `run_context' in one of the threads in the
- thread pool. Returns FALSE if the thread pool is being freed. If
- there are no free threads left in the pool this will queue the
- the `run' and will call it once a thread becomes free.
+ o Add '%@' format to silc_snprintf functions.
+ (***DONE)
- If `completion' is non-NULL it will be called to indicate completion
- of the `run' function. If `schedule' is non-NULL the `completion'
- will be called through the scheduler in the main thread. If it is
- NULL the `completion' is called directly from the thread after the
- `run' has returned. */
- SilcBool silc_thread_pool_run(SilcThreadPool tp,
- SilcSchedule schedule,
- SilcThreadPoolFunc run,
- void *run_context,
- SilcThreadPoolFunc completion,
- void *completion_context);
+ o SILC Tls (Thread-local storage) API to lib/silcutil/silcthread.[ch].
+ (***DONE)
- /* Modify the amount of maximum threads of the pool. */
- void silc_thread_pool_set_max_threads(SilcThreadPool tp,
- SilcUInt32 max_threads);
+ o Change silc_gettimeofday on Unix to use clock_gettime with REALTIME
+ clock if it is available, otherwise use gettimeofday(). (***DONE)
- /* Returns the amount of maximum size the pool can grow. */
- SilcUInt32 silc_thread_pool_get_max_threads(SilcThreadPool tp);
+ o Generic SilcResult that includes all possible status and
+ error conditions and generic errno API. (***DONE)
- /* Returns the amount of free threads in the pool currently. */
- SilcUInt32 silc_thread_pool_get_free_threads(SilcThreadPool tp);
+ (o Structured log messages to Log API. Allows machine readable log
+ messages. Would allow sending of any kind of data in a log message.) maybe
- /* Stops all free and started threads. The minumum amount of threads
- specified to silc_thread_pool_alloc always remains. */
- void silc_thread_pool_stop_free_threads(SilcThreadPool tp);
+ (o Change some stream routines (like socket stream API) to accept ANY
+ stream and use silc_stream_get_root to get the socket stream from the
+ given stream. This will make various stream APIs more easier to use
+ when user doesn't have to dig up the correct stream.
- o Fast mutex implementation. Fast rwlock implementation. Mutex and
- rwlock implementation using atomic operations.
+ Add silc_stream_get_root and add get_root stream operation. It
+ returns the root of the stream or NULL if stream doesn't have root.) maybe
- o Compression routines are missing. The protocol supports packet
+ (o Compression routines are missing. The protocol supports packet
compression thus it must be implemented. SILC Zip API must be
- defined.
+ defined.) maybe
- (o Generic SilcStatus or SilcResult that includes all possible status and
- error conditions, including those of SILC protocol. Though, the SILC
- protocol related status (currently in silcstatus.h) cannot be in
- runtime library) maybe
+ (o SilcIpAddr abstraction. Ipv4 and Ipv6 support to the abstaction.)
+ maybe
(o SILC specific socket creation/closing routines to silcnet.h, wrappers
to all send(), recv(), sendto() etc. Bad thing is that we'd have to
define all socket options, sockaddrs, etc.) maybe
- (o mmap) maybe
+ (o Fast mutex implementation. Fast rwlock implementation. Mutex and
+ rwlock implementation using atomic operations.) not for now.
lib/silcutil/symbian/
like silc_debug etc. They won't work on EPOC. Perhaps logging
and debugging is to be disabled on EPOC. The logging currently works
by it cannot be controlled, same with debugging.
-
-
-SFTP Library, lib/silcsftp/
-===========================
-
- o Read prefetch (read-ahead, reading ahead of time). Maybe if this can
- be done easily.
-
-
-SKR Library, lib/silcskr/
-=========================
-
- o Add fingerprint as search constraint.
-
- o Add OpenPGP support. Adding, removing, fetching PGP keys. (Keyring
- support?)
-
- o Add support for importing public keys from a directory and/or from a
- file. Add support for exporting the repository (different formats for
- different key types?).
-
- o Change the entire silc_skr_find API. Remove SilcSKRFind and just simply
- add the find constraints as variable argument list to silc_skr_find, eg:
-
- silc_skr_find(skr, schedule, callback, context,
- SILC_SKR_FIND_PUBLIC_KEY, public_key,
- SILC_SKR_FIND_COUNTRY, "FI",
- SILC_SKR_FIND_USAGE, SILC_SKR_USAGE_AUTH,
- SILC_SKR_FIND_END);
-
- NULL argument would be ignored and skipped.
-
- o Add OR logical rule in addition of the current default AND, eg:
-
- // Found key(s) MUST have this public key AND this country.
- silc_skr_find(skr, schedule, callback, context,
- SILC_SKR_FIND_RULE_AND,
- SILC_SKR_FIND_PUBLIC_KEY, public_key,
- SILC_SKR_FIND_COUNTRY, "FI",
- SILC_SKR_FIND_END);
-
- // Found key(s) MUST have this public key OR this key context
- silc_skr_find(skr, schedule, callback, context,
- SILC_SKR_FIND_RULE_OR,
- SILC_SKR_FIND_PUBLIC_KEY, public_key,
- SILC_SKR_FIND_CONTEXT, key_context,
- SILC_SKR_FIND_END);
-
-
-Crypto Library, lib/silccrypt/
-==============================
-
- o Add fingerprint to SilcSILCPublicKey and retrieval to silcpk.h, and
- possibly to silcpkcs.h.
-
- /* Return fingerprint of the `public_key'. Returns also the algorithm
- that has been used to make the fingerprint. */
- const unsigned char *
- silc_pkcs_get_fingerprint(SilcPublicKey public_key,
- const char **hash_algorithm,
- SilcUInt32 *fingerprint_len);
-
- o Change SILC PKCS API to asynchronous, so that accelerators can be used.
- All PKCS routines should now take callbacks as argument and they should
- be delivered to SilcPKCSObject and SilcPKCSAlgorithm too.
-
- /* Signature computation callback */
- typedef void (*SilcPKCSSignCb)(SilcBool success,
- const unsigned char *signature,
- SilcUInt32 signature_len,
- void *context);
-
- /* Signature verification callback */
- typedef void (*SilcPKCSVerifyCb)(SilcBool success, void *context);
-
- /* Encryption callback */
- typedef void (*SilcPKCSEncryptCb)(SilcBool success,
- const unsigned char *encrypted,
- SilcUInt32 encrypted_len,
- void *context);
-
- /* Decryption callback */
- typedef void (*SilcPKCSDecryptCb)(SilcBool success,
- const unsigned char *decrypted,
- SilcUInt32 decrypted_len,
- void *context);
-
- Either add new _async functions or add the callbacks to existing API
- and if the callback is NULL then the API is not async and if provided
- it may be async. For example;
-
- SilcBool silc_pkcs_sign(SilcPrivateKey private_key,
- unsigned char *src, SilcUInt32 src_len,
- unsigned char *dst, SilcUInt32 dst_size,
- SilcUInt32 *dst_len,
- SilcBool compute_hash, SilcHash hash,
- SilcPKCSSignCb async_sign,
- void *async_sign_context);
-
- (if this is done then there's no reason why the buffers in the
- callbacks cannot be the ones user gives here) or allow only async:
-
- SilcBool silc_pkcs_sign(SilcPrivateKey private_key,
- unsigned char *src, SilcUInt32 src_len,
- SilcBool compute_hash, SilcHash hash,
- SilcPKCSSignCb async_sign,
- void *async_sign_context);
-
- or add new:
-
- SilcBool silc_pkcs_sign_async(SilcPrivateKey private_key,
- unsigned char *src, SilcUInt32 src_len,
- SilcBool compute_hash, SilcHash hash,
- SilcPKCSSignCb async_sign,
- void *async_sign_context);
-
- o Change PKCS Algorithm API to take SilcPKCSAlgorithm as argument to
- encrypt, decrypt, sign and verify functions. We may need to for exmaple
- check the alg->hash, supported hash functions. Maybe deliver it also
- to all other functions in SilcPKCSAlgorithm to be consistent.
-
- o Add DSS support. Take implementation from Tom or make it yourself.
-
- o Implement the defined SilcDH API. The definition is in
- lib/silccrypt/silcdh.h. Make sure it is asynchronous so that it can
- be accelerated. Also take into account that it could use elliptic
- curves.
-
- o ECDSA and ECDH
-
- o All cipher, hash, hmac etc. allocation routines should take their name
- in as const char * not const unsigned char *.
-
-
-SILC Accelerator Library
-========================
-
- o SILC Accelerator API. Provides generic way to use different kind of
- accelerators. Basically implements SILC PKCS API so that SilcPublicKey
- and SilcPrivateKey can be used but they call the accelerators.
-
- Something in the lines of (preliminary):
-
- /* Register accelerator to system */
- SilcBool silc_acc_register(const SilcAccelerator acc);
-
- /* Unregister accelerator */
- SilcBool silc_acc_unregister(const SilcAccelerator acc);
-
- /* Find existing accelerator. `name' is accelerator's name. Varags
- are optional accelerator specific parameteres. */
- SilcAccelerator silc_acc_find(const char *name, ...);
-
- silc_acc_find("softacc", "min_threads", 2, "max_threads", 8, NULL);
-
- /* Return accelerator's displayable name */
- const char *silc_ac_get_display_name(SilcAccelerator acc);
-
- /* Accelerate `public_key'. Return accelerated public key. */
- SilcPublicKey silc_acc_public_key(SilcAccelerator acc,
- SilcPublicKey public_key);
-
- /* Accelerate `private_key'. Returns accelerated private key. */
- SilcPrivateKey silc_acc_private_key(SilcAccelerator acc,
- SilcPrivateKey private_key);
-
- /* Return the underlaying public key */
- SilcPublicKey silc_acc_get_public_key(SilcAccelerator acc,
- SilcPublicKey public_key);
-
- /* Return the underlaying private key */
- SilcPrivateKey silc_acc_get_private_key(SilcAccelerator acc,
- SilcPrivateKey private_key);
-
- typedef enum {
- SILC_ACCELERATOR_PKCS,
- SILC_ACCELERATOR_DH,
- SILC_ACCELERATOR_CIPHER,
- SILC_ACCELERATOR_HASH
- } SilcAcceleratorType;
-
- typedef struct SilcAcceleratorObject {
- const char *name; /* Accelerator's name */
- const char *display_name; /* Displayable name */
- union {
- struct {
- SilcPKCSObject *pkcs; /* Accelerator PKCS*/
- SilcPKCSAlgorithm *algorithm; /* Accelerator Alg, may be NULL */
- } pkcs;
-
- SilcDHObject *dh;
- SilcCipherObject *cipher;
- SilcHashObject *hash;
- } u;
-
- SilcAcceleratorType type; /* Accelerator type */
- } *SilcAccelerator, SilcAcceleratorStruct;
-
- SilcPublicKey->SilcSILCPublicKey->RsaPublicKey accelerated as:
- SilcPublicKey->SilcAcceleratorSoftware->SilcSILCPublicKey->RsaPublicKey or
- SilcPublicKey->SilcAcceleratorPublicKey->SilcAcceleratorSoftware->
- SilcSILCPublicKey->RsaPublicKey
-
- The former one if u.pkcs.algorithm == NULL.
-
- o Implement software accelerator. It is a thread pool system where the
- public key and private key operations are executed in threads.
-
- This implements SilcPKCSAlgorithm (and SilcPKCSObject if needed) that
- implements the thread acclerated system.
-
- (o Symmetric key cryptosystem acceleration? They are always sycnhronouos
- even with hardware acceleration so the crypto API shouldn't require
- changes.) maybe
-
-
-lib/silcmath
-============
-
- o Import TFM. Talk to Tom to add the missing functions. Use TFM in
- client and client library, but TMA in server, due to the significantly
- increased memory consumption with TFM, and the rare need for public
- key operations in server.
-
- We want TFM's speed but not TFM's memory requirements. Talk to Tom
- about making the TFM mp dynamic just as it is in LTM.
-
- o The SILC MP API function must start returning indication of success
- and failure of the operation.
-
- o Do SilcStack support for silc_mp_init, silc_mp_init_size and other
- any other MP function (including utility ones) that may allocate
- memory.
-
- o All utility functions should be made non-allocating ones.
-
-
-SILC XML Library, lib/silcxml/
-==============================
-
- o SILC XML API (wrapper to expat). The SILC XML API should follow and
- resemble Simple API for XML (SAX).
-
-
-lib/silcske/silcske.[ch]
-========================
-
- o Ratelimit to UDP/IP transport for incoming packets.
-
-
-lib/silcasn1
-============
-
- o Negative integer encoding is missing, add it.
-
- o SILC_ASN1_CHOICE should perhaps return an index what choice in the
- choice list was found. Currently it is left for caller to figure out
- which choice was found.
-
- o SILC_ASN1_NULL in decoding should return SilcBool whether or not
- the NULL was present. It's important when it's SILC_ASN1_OPTIONAL
- and we need to know whether it was present or not.
-
-
-lib/silcpgp
-===========
-
- o OpenPGP certificate support, allowing the use of PGP public keys
- in SILC.
-
-
-lib/silcssh
-===========
-
- o SSH2 public key/private key support, allowing the use of SSH2 keys
- in SILC. RFC 4716.
-
-
-lib/silcpkix
-============
-
- o PKIX implementation
-
-
-lib/silcserver
-==============
-
- o (Re)write commands/command replys.
-
- o (Re)write notify handling.
-
- o The SERVER_SIGNOFF notify handing is not optimal, because it'll
- cause sending of multiple SIGNOFF notify's instead of the one
- SERVER_SIGNOFF notify that the server received. This should be
- optimized so that the only SERVER_SIGNOFF is sent and not
- SIGNOFF of notify at all (using SIGNOFF takes the idea about
- SERVER_SIGNOFF away entirely).
-
- o Another SERVER_SIGNOFF opt/bugfix: Currently the signoff is
- sent to a client if it is on same channel as the client that
- signoffed. However, the entire SERVER_SIGNOFF list is sent to
- the client, ie. it may receive clients that was not on the
- same channel. This is actually against the specs. It must be
- done per channel. It shouldn't receive the whole list just
- because one client happened to be on same channel.
-
- o Add reference counters to all Silc*Entry structures
-
- o SERVICEs support (plugin, SIM)
-
- o If client's public key is saved in the server (and doing public key
- authentication) then the hostname and the username information could
- be taken from the public key. Should be a configuration option!
-
- o Add a timeout to handling incoming JOIN commands. It should be
- enforced that JOIN command is executed only once in a second or two
- seconds. Now it is possible to accept n incoming JOIN commands
- and process them without any timeouts. THis must be employed because
- each JOIN command will create and distribute the new channel key
- to everybody on the channel.
-
- o Related to above. If multiple JOINs are received in sequence perhaps
- new key should be created only once, if the JOINs are handeled at the same
- time. Now we create multiple keys and never end up using them because
- many JOINs are processed at the same time in sequence. Only the last
- key ends up being used.
-
- o The CMODE cipher & hmac change problem (#101).