-TODO for 1.2 And Beyond
-=======================
+SILC Runtime Toolkit 1.2 and beyond
+===================================
NOTE: Any item that doesn't have (***DONE) in it, isn't done yet. The
(***TESTING NEEDED) means that the item has been done but not yet properly
to silc-devel mailing list or appear on 'silc' channel on SILCNet.
-General
-=======
-
- o Create apps/tutorial containing various Toolkit API tutorials.
-
- o The Toolkit split. The Toolkit is to be splitted in parts. How many
- parts and what the parts are isn't decided yet. Each part is a separate
- software package. Current thinking is of the following:
-
- SILC Toolkit SILC protocol, client and server library
- SILC Runtime Toolkit runtime library
- SILC Crypto Toolkit crypto, asn1, math, skr, pgp, etc.
-
- The rationale for this is of course that other than SILC projects
- might like to use the various libraries SILC Toolkit provides, but
- naturally they don't want the bloat of SILC protocol related stuff.
-
- The Runtime library in SILC Toolkit is a general purpose runtime library,
- like Glib and APR are. The runtime library is to be developed further
- to provide alternative to Glib and APR.
-
- The Crypto library in SILC Toolkit is a general purpose crypto library
- providing pretty nice APIs compared to many other crypto libraries,
- especially OpenSSL. The Crypto library is to be developed further
- to include support for OpenPGP, X.509 and SSH2.
-
-
-lib/silccore
-============
-
- o SILC_PACKET_FLAG_ACK support. Implement ACK packet and packet payload
- to silcpacket.c.
-
- o All payload encoding routines should take SilcStack as argument.
-
- o Remove SilcCommandCb from silccommand.h.
+Runtime library, lib/silcutil/
+==============================
- o All payload test routines into lib/silccore/tests/.
+ o Add SILC Zip API, compression.
+ o file removing, chmod, rmmod, etc. chdir, rmdir, stat, etc. to
+ lib/silcutil/silcfileutil.h.
-lib/silcclient, The Client Library
-==================================
+ o mmap API
- o UDP SILC connection support to SILC server
+ o Multicast support.
- o Giving WHOIS for nick that doesn't exist should remove any same
- named entries from the client cache.
+ o Implement silc_get_num_cpu.
- o peer-to-peer private messages
+ o silc_file_readfile and silc_file_writefile should perhaps do the thing
+ with mmap, as it's probably a bit faster.
- o Private message key request notification to application. See XXX in
- client_prvmsg.c.
+ o silc_malloc et. al. to respect --with-alignment.
- o in JOIN notify handle resolving that timedout. Currently the user is
- never joined the channel if this happens. What to do if message is
- received from user that hasn't been resolved/joined?
+ o Fix universal time decoding (doesn't accept all formats) in silctime.c.
- o Add the SilcStream (socket stream) from the SilcPacketStream and
- SilcSocket from the socket stream to SilcClientConnection for easier
- access to them for programmers. Currently these have to be digged up
- from the packet stream.
+ o Additional scheduler changes: optimize silc_schedule_wakeup. Wakeup
+ only if the scheduler is actually waiting something. If it is
+ delivering tasks wakeup is not needed.
- o Connection option that attemps to connect to remot host with various
- different mechanisms: UDP 706, TCP 706, TCP 80, TCP 443, UDP 7706 and
- TCP 7706. This is the so called hole punching mechanism.
+ o silc_stringprep to non-allocating version.
- o Message ACKing support.
+ o Add builtin SOCKS and HTTP Proxy support, well the SOCKS at least.
+ SILC currently supports SOCKS4 and SOCKS5 but it needs to be compiled
+ in separately.
- o in /cmode and /cumode with +r, maybe the public key and private key
- could be just some "string", which would then match to "string.pub" and
- "string.prv".
+ o Add silc_xml_parse_stream to parse SilcStream XML stream.
- o If the SILC Events (see below) are implemented, perhaps client library
- should provide events so that application developer has a choice of
- developing the SILC app with callbacks or with events.
+ o SILC XML API (wrapper to expat). (***DONE)
- o Ability to recover from rekey errors, at least try to.
+ o Bring silchttp HTTP server library to SRT. (***DONE)
+ o Simple SILC Rand API for pseudo-random numbers. (***DONE)
-Runtime library, lib/silcutil/
-==============================
+ o Add directory opening/traversing functions (***DONE, TODO Windows & Symbian)
- o Fix universal time decoding (doesn't accept all formats) in silctime.c.
+ o regex from /lib/contrib to lib/silcutil, define SILC Regex API. (***DONE)
o Add functions to manipulate environment variables. (***DONE)
SIM library (lib/silcsim) and introduces generic library). Add this
to lib/silcutil/silcdll.[ch]. (***TESTING NEEDED WIN32, TODO Symbian)
- o Add directory opening/traversing functions
-
- o silc_getopt routines
-
- o Add silc_stream_get_root and add get_root stream operation. It
- returns the root of the stream or NULL if stream doesn't have root.
-
- o Change some stream routines (like socket stream API) to accept ANY
- stream and use silc_stream_get_root to get the socket stream from the
- given stream. This will make various stream APIs more easier to use
- when user doesn't have to dig up the correct stream.
+ o silc_getopt routines (***DONE)
o The SILC Event signals. Asynchronous events that can be created,
connected to and signalled. Either own event routines or glued into
- SilcSchedule:
-
- SilcTask silc_schedule_task_add_event(SilcSchedule schedule,
- const char *event, ...);
- SilcBool silc_schedule_event_connect(SilcSchedule schedule,
- const char *event,
- SilcTaskCallback event_callback,
- void *context);
- SilcBool silc_schedule_event_signal(SilcSchedule schedule,
- const char *event, ...);
-
- Example:
- silc_schedule_task_add_event(schedule, "connected",
- SILC_PARAM_UI32_INT,
- SILC_PARAM_BUFFER,
- SILC_PARAM_END);
- silc_schedule_event_connect(schedule, "connected", connected_cb, ctx);
- silc_schedule_event_signal(schedule, "connected", integer, buf,
- SILC_PARAM_END);
- SILC_TASK_CALLBACK(connected_cb)
- {
- FooCtx ctx = context;
- va_list args;
- SilcUInt32 integer;
- SilcBuffer buf;
-
- va_start(args, context);
- integer = va_arg(args, SilcUInt32);
- buf = va_arg(args, SilcBuffer);
- va_end(args);
- ...
- }
-
- Problems: Events would be SilcSchedule specific, and would not work on
- multi-thread/multi-scheduler system. The events should be copyable
- between schedulers. Another problem is the signal delivery. Do we
- deliver them synchronously possibly from any thread to any other thread
- or do we deliver them through the target schedulers. If we use the
- schedulers then signalling would be asynchronous (data must be
- duplicated and later freed) which is not very nice.
+ SilcSchedule. (***DONE)
o If the event signals are added, the SILC_PARAM_* stuff needs to be
- moved from silcbuffmt.h to silctypes.h or something similar.
+ moved from silcbuffmt.h to silctypes.h or something similar. (***DONE)
o In case the SILC Events are done we shall create a new concept of
parent and child SilcSchedule's. When new SilcSchedule is created a
would be linked and could be accessed from any of the schedulers.
It should be possible to retrieve the parent and enumerate all children
from any of the schedulers.
-
- SilcSchedule silc_schedule_init(int max_tasks, void *app_context,
- SilcSchedule parent);
- SilcSchedule silc_schedule_get_parent(SilcSchedule schedule);
-
- o Additional scheduler changes: optimize silc_schedule_wakeup. Wakeup
- only if the scheduler is actually waiting something. If it is
- delivering tasks wakeup is not needed.
-
- o Structured log messages to Log API. Allows machine readable log
- messages. Would allow sending of any kind of data in a log message.
+ (***DONE)
o Base64 to an own API (***DONE)
o Timer API (***DONE)
- o Add builtin SOCKS and HTTP Proxy support, well the SOCKS at least.
- SILC currently supports SOCKS4 and SOCKS5 but it needs to be compiled
- in separately.
-
- o silc_stringprep to non-allocating version.
-
o silc_hash_table_replace -> silc_hash_table_set. Retain support for
silc_hash_table_replace as macro. (***DONE)
o Thread pool API. Add this to lib/silcutil/silcthread.[ch]. (***DONE)
- o Fast mutex implementation. Fast rwlock implementation. Mutex and
- rwlock implementation using atomic operations.
-
- o Compression routines are missing. The protocol supports packet
- compression thus it must be implemented. SILC Zip API must be
- defined.
-
o Add new functions to SilcStack API in lib/silcutil/silcstack.[ch]. Add
silc_stack_[set|get]_alignment. It defines the default alignment used
when allocating memory from stack. It can be used to specify special
_ua unaligned memory allocation routines. Remove unaligned memory
allocation possibility. (***DONE)
- o silc_stack_alloc shouldn't require multiple by 8 size argument, it
- should figure it out itself.
+ o silc_stack_alloc shouldn't require multiple by 8 size argument, it
+ should figure it out itself. (***DONE)
- o silc_malloc et. al. to respect --with-alignment.
+ o Add '%@' format to silc_snprintf functions.
+ (***DONE)
- o Add '%@' format to silc_snprintf functions. It marks for external
- rendering function of following type:
+ o SILC Tls (Thread-local storage) API to lib/silcutil/silcthread.[ch].
+ (***DONE)
- /* Snprintf rendering function. The `data' is rendered into a string
- and allocated string is returned. If NULL is returned the
- rendering is skipped and ignored. If the returned string does
- not fit to the destination buffer it may be truncated. */
- typedef char *(*SilcSnprintfRender)(void *data);
+ o Change silc_gettimeofday on Unix to use clock_gettime with REALTIME
+ clock if it is available, otherwise use gettimeofday(). (***DONE)
- It can work like following:
+ o Generic SilcResult that includes all possible status and
+ error conditions and generic errno API. (***DONE)
- char *id_renderer(void *data)
- {
- char tmp[32];
- id_to_str(tmp, sizeof(tmp), (SilcID *)data);
- return strdup(tmp);
- }
+ (o Structured log messages to Log API. Allows machine readable log
+ messages. Would allow sending of any kind of data in a log message.) maybe
- silc_snprintf(buf, sizeof(buf), "Client ID %@", id_renderer, client_id);
- (***DONE)
+ (o Change some stream routines (like socket stream API) to accept ANY
+ stream and use silc_stream_get_root to get the socket stream from the
+ given stream. This will make various stream APIs more easier to use
+ when user doesn't have to dig up the correct stream.
- o Change silc_gettimeofday on Unix to use clock_gettime with REALTIME
- clock if it is available, otherwise use gettimeofday(). (***DONE)
+ Add silc_stream_get_root and add get_root stream operation. It
+ returns the root of the stream or NULL if stream doesn't have root.) maybe
- (o SilcIpAddr abstraction. Ipv4 and Ipv6 support to the abstaction.)
+ (o SilcIpAddr abstraction. Ipv4 and Ipv6 support to the abstaction.)
maybe
- (o Generic SilcStatus or SilcResult that includes all possible status and
- error conditions, including those of SILC protocol. Though, the SILC
- protocol related status (currently in silcstatus.h) cannot be in
- runtime library) maybe
-
(o SILC specific socket creation/closing routines to silcnet.h, wrappers
to all send(), recv(), sendto() etc. Bad thing is that we'd have to
define all socket options, sockaddrs, etc.) maybe
- (o mmap) maybe
+ (o Fast mutex implementation. Fast rwlock implementation. Mutex and
+ rwlock implementation using atomic operations.) not for now.
+
+
+Windows Support
+===============
-lib/silcutil/symbian/
-=====================
+Symbian OS Support
+==================
o Something needs to be thought to the logging globals as well,
like silc_debug etc. They won't work on EPOC. Perhaps logging
and debugging is to be disabled on EPOC. The logging currently works
- by it cannot be controlled, same with debugging.
-
-
-SFTP Library, lib/silcsftp/
-===========================
-
- o Read prefetch (read-ahead, reading ahead of time). Maybe if this can
- be done easily.
-
-
-SKR Library, lib/silcskr/
-=========================
-
- o Add fingerprint as search constraint.
-
- o Add OpenPGP support. Adding, removing, fetching PGP keys. (Keyring
- support?)
-
- o Add support for importing public keys from a directory and/or from a
- file. Add support for exporting the repository (different formats for
- different key types?).
-
- o Change the entire silc_skr_find API. Remove SilcSKRFind and just simply
- add the find constraints as variable argument list to silc_skr_find, eg:
-
- silc_skr_find(skr, schedule, callback, context,
- SILC_SKR_FIND_PUBLIC_KEY, public_key,
- SILC_SKR_FIND_COUNTRY, "FI",
- SILC_SKR_FIND_USAGE, SILC_SKR_USAGE_AUTH,
- SILC_SKR_FIND_END);
-
- NULL argument would be ignored and skipped.
-
- o Add OR logical rule in addition of the current default AND, eg:
-
- // Found key(s) MUST have this public key AND this country.
- silc_skr_find(skr, schedule, callback, context,
- SILC_SKR_FIND_RULE_AND,
- SILC_SKR_FIND_PUBLIC_KEY, public_key,
- SILC_SKR_FIND_COUNTRY, "FI",
- SILC_SKR_FIND_END);
-
- // Found key(s) MUST have this public key OR this key context
- silc_skr_find(skr, schedule, callback, context,
- SILC_SKR_FIND_RULE_OR,
- SILC_SKR_FIND_PUBLIC_KEY, public_key,
- SILC_SKR_FIND_CONTEXT, key_context,
- SILC_SKR_FIND_END);
-
- o SilcStack to SKR API.
-
-
-Crypto Library, lib/silccrypt/
-==============================
-
- o Add silc_crypto_init and silc_crypto_uninit. The _init should take
- SilcStack that will act as global memory pool for all of crypto
- library. It should not be necessary anymore to separately register
- default ciphers, HMACs, etc, the _init would do that. However, if
- user after _init calls silc_pkcs_register, for example, it would take
- preference over the default once, ie. user can always dictate the
- order of algorithms. (***DONE)
-
- o Add fingerprint to SilcSILCPublicKey and retrieval to silcpk.h, and
- possibly to silcpkcs.h.
-
- /* Return fingerprint of the `public_key'. Returns also the algorithm
- that has been used to make the fingerprint. */
- const unsigned char *
- silc_pkcs_get_fingerprint(SilcPublicKey public_key,
- const char **hash_algorithm,
- SilcUInt32 *fingerprint_len);
-
- o Change SILC PKCS API to asynchronous, so that accelerators can be used.
- All PKCS routines should now take callbacks as argument and they should
- be delivered to SilcPKCSObject and SilcPKCSAlgorithm too. (***DONE)
-
- o Change PKCS Algorithm API to take SilcPKCSAlgorithm as argument to
- encrypt, decrypt, sign and verify functions. We may need to for exmaple
- check the alg->hash, supported hash functions. Maybe deliver it also
- to all other functions in SilcPKCSAlgorithm to be consistent. (***DONE)
-
- o Add DSA support to SILC public key.
-
- o Add DSS support. (***DONE)
-
- o Implement the defined SilcDH API. The definition is in
- lib/silccrypt/silcdh.h. Make sure it is asynchronous so that it can
- be accelerated. Also take into account that it could use elliptic
- curves.
-
- o Add ECDSA support.
-
- o Add ECDH support.
-
- o AES CBC is missing proper alignment code (see silc_1_1_branch).
-
- o All cipher, hash, hmac etc. allocation routines should take their name
- in as const char * not const unsigned char *. (***DONE)
-
-
-SILC Accelerator Library
-========================
-
- o SILC Accelerator API. Provides generic way to use different kind of
- accelerators. Basically implements SILC PKCS API so that SilcPublicKey
- and SilcPrivateKey can be used but they call the accelerators.
- (***DONE)
-
- o Implement software accelerator. It is a thread pool system where the
- public key and private key operations are executed in threads.
- (***DONE)
-
- o Add init options to SilcAcceleratorObject as a SilcAcceleratorOption
- structure. Each accelerator defines the options that they support and
- can be retrieved from the SilcAccelerator with silc_acc_get_options.
- The format must also be machine parseable. The structure can be of the
- following format:
-
- typedef struct SilcAcceleratorOptionStruct {
- const char *option; /* Option name */
- const char *display_name; /* Option displayable name */
- SilcParamType type; /* Option data format */
- } *SilcAcceleratorOption;
-
- For software accelerator it could be for example:
-
- { "min_threads", "Minimum threads", SILC_PARAM_UINT32 },
- { "max_threads", "Maximum threads", SILC_PARAM_UINT32 },
-
- The accelerator itself doesn't have to use the option structure to
- parse the options if not wanted. It is defined for the caller so
- they can learn the supported options in a well defined way.
-
- o Diffie-Hellman acceleration
-
- (o Symmetric key cryptosystem acceleration? They are always sycnhronouos
- even with hardware acceleration so the crypto API shouldn't require
- changes.) maybe
-
-
-lib/silcmath
-============
-
- o Import TFM. We want TFM's speed but its memory requirements are
- just too much. By default it uses large pre-allocated tables which
- will eat memory when there are thousands of public keys in system.
- We probably want to change TFM's fp_int dynamic so that a specific
- size can be allocated for the int. We could have two new functions:
-
- SilcBool silc_mp_init_size(SilcMPInt *mp, SilcUInt32 bit_size);
- SilcBool silc_mp_sinit_size(SilcStack stack, SilcMPInt *mp,
- SilcUInt32 bit_size);
-
- Which by default allocates `bit_size' bits instead of some default
- value. silc_mp_init would allocate the default FP_SIZE with TFM
- and do normal init with TMA and GMP. _init_size with TMA and GMP
- would be same as _init.
-
- o Add AND, OR and XOR support to TFM or ask Tom to do it.
-
- o The SILC MP API function must start returning indication of success
- and failure of the operation.
-
- o Do SilcStack support for silc_mp_init, silc_mp_init_size and other
- any other MP function (including utility ones) that may allocate
- memory.
-
- o Prime generation progress using callback instead of printing to
- stdout.
-
- o All utility functions should be made non-allocating ones.
-
-
-SILC XML Library, lib/silcxml/
-==============================
-
- o SILC XML API (wrapper to expat). Look at the expat API and simplify
- it. The SILC XML API should have at most 8-10 API functions. It should
- be possible to create full XML parser with only one function. And, it
- should be possible to have a function that is able to parse an entire
- XML document. It should also have a parser function to be able to
- parse a stream of XML data (SilcStream). It MUST NOT have operations
- that require multiple function calls to be able to execute that one
- operation (like creating parser).
-
-
-lib/silcske/silcske.[ch]
-========================
-
- o Ratelimit to UDP/IP transport for incoming packets.
-
-
-lib/silcasn1
-============
-
- o Negative integer encoding is missing, add it.
-
- o SILC_ASN1_CHOICE should perhaps return an index what choice in the
- choice list was found. Currently it is left for caller to figure out
- which choice was found. (***DONE)
-
- o SILC_ASN1_NULL in decoding should return SilcBool whether or not
- the NULL was present. It's important when it's SILC_ASN1_OPTIONAL
- and we need to know whether it was present or not. (***DONE)
-
-
-lib/silcpgp
-===========
-
- o OpenPGP certificate support, allowing the use of PGP public keys.
-
-
-lib/silcssh
-===========
-
- o SSH2 public key/private key support, allowing the use of SSH2 keys.
- RFC 4716. (***DONE)
-
-
-lib/silcpkix
-============
-
- o PKIX implementation
-
-
-apps/silcd
-==========
-
- o Deprecate the old server. Write interface for the new lib/silcserver
- server library. The interface should work on Unix/Linux systems.
-
- o Consider deprecating also the old config file format and use XML
- istead. This should require SILC XML API implementation first.
-
- o The configuration must support dynamic router and server connections.
- The silcd must work without specifying any servers or routers to
- connect to.
-
- o The configuration must support specifying whether the server is
- SILC Server or SILC Router. This should not be deduced from the
- configuration as it was in < 1.2.
-
- o The configuration must support specifying the ciphers and hmacs and
- their order so that user can specify which algorithms take preference.
-
-
-lib/silcserver
-==============
-
- o Rewrite the entire server. Deprecate apps/silcd as the main server
- implementation and create lib/silcserver/. It is a platform
- independent server library. The apps/silcd will merely provide a
- a simple interface for the library.
-
- o Write the SILC Server library extensively using SILC FSM.
-
- o Server library must support multiple networks. This means that one
- server must be able to create multiple connections that each reach
- different SILC network. This means also that all cache's etc. must
- be either connection-specific or network-specific.
-
- o Library must support dynamic router and server connections. This means
- that connections are create only when they are needed, like when someone
- says JOIN foo@foo.bar.com or WHOIS foobar@silcnet.org.
-
- o Library must support server-to-server connections even though protocol
- prohibits that. The responder of the connection should automatically
- act as a router. The two servers create an own, isolated, SILC network.
- To be used specifically with dynamic connections.
-
- o Library must support multiple threads and must be entirely thread safe.
-
- o Library must have support for SERVICE command.
-
- o Both UDP and TCP support for incoming connecetions. Maintaining long
- term UDP sessions.
-
- o The server must be able to run behind NAT device. This means that
- Server ID must be based on public IP instead of private IP.
-
- o The following data must be in per-connection context: client id cache,
- server id cache, channel id cache, all statistics must be
- per-connection.
-
- o The following data must be in per-thread context: command context
- freelist/pool, pending commands, random number generator.
-
- o Do inccoming packet processing in an own FSM thread in the
- server-threads FSM. Same as in client library.
-
- o Binding to other ports than 706 too. To allow easier traversing
- through NATs and firewalls server should bind to 80, 443 and 7706
- by default (at least try to bind). Connections must work normally
- even if they were established to some other port other than 706.
-
- Connection option that attemps to connect to remot server with various
- different mechanisms: UDP 706, TCP 706, TCP 80, TCP 443, UDP 7706 and
- TCP 7706. This is the so called hole punching mechanism.
-
- o Ability to recover from rekey errors, at least try to.
-
- o Reference count all Silc*Entry structures.
-
- Some issues that must be kept in mind from 1.0 and 1.1 silcd's:
-
- o The server and router software MUST work out of the box. After
- installation the server must not require any configuration to run the
- most basic working configuration. No defining IP addresses, etc.
- The server must work just by running it.
-
- o The SERVER_SIGNOFF notify handing is not optimal, because it'll
- cause sending of multiple SIGNOFF notify's instead of the one
- SERVER_SIGNOFF notify that the server received. This should be
- optimized so that the only SERVER_SIGNOFF is sent and not
- SIGNOFF of notify at all (using SIGNOFF takes the idea about
- SERVER_SIGNOFF away entirely).
-
- o Another SERVER_SIGNOFF opt/bugfix: Currently the signoff is
- sent to a client if it is on same channel as the client that
- signoffed. However, the entire SERVER_SIGNOFF list is sent to
- the client, ie. it may receive clients that was not on the
- same channel. This is actually against the specs. It must be
- done per channel. It shouldn't receive the whole list just
- because one client happened to be on same channel.
-
- o If client's public key is saved in the server (and doing public key
- authentication) then the hostname and the username information could
- be taken from the public key. Should be a configuration option!
+ by it cannot be controlled, same with debugging. SILC Global API
+ MUST be used with all globals on Symbian.
- o Add a timeout to handling incoming JOIN commands. It should be
- enforced that JOIN command is executed only once in a second or two
- seconds. Now it is possible to accept n incoming JOIN commands
- and process them without any timeouts. THis must be employed because
- each JOIN command will create and distribute the new channel key
- to everybody on the channel.
+ o gethostname() returns "Function not implemented". Others may return
+ the same. We should fix that probably to use RHostResolver and
+ GetHostName().
- o Related to above. If multiple JOINs are received in sequence perhaps
- new key should be created only once, if the JOINs are handeled at the same
- time. Now we create multiple keys and never end up using them because
- many JOINs are processed at the same time in sequence. Only the last
- key ends up being used.
+ o silc_thread_exit should call User::Exit().