1 TODO for 1.2 And Beyond
2 =======================
4 NOTE: Any item that doesn't have (***DONE) in it, isn't done yet. The
5 (***TESTING NEEDED) means that the item has been done but not yet properly
8 NOTE: A TODO entry does not mean that it is ever going to be done. Some
9 of the entries may be just ideas, good, bad or ugly. If you want to work
10 on some of the TODO entries simply let us know about it by dropping a note
11 to silc-devel mailing list or appear on 'silc' channel on SILCNet.
17 o Create apps/tutorial containing various Toolkit API tutorials.
19 o The Toolkit split. The Toolkit is to be splitted in parts. How many
20 parts and what the parts are isn't decided yet. Each part is a separate
21 software package. Current thinking is of the following:
23 SILC Toolkit SILC protocol, client and server library
24 SILC Runtime Toolkit runtime library
25 SILC Crypto Toolkit crypto, asn1, math, skr, pgp, etc.
27 The rationale for this is of course that other than SILC projects
28 might like to use the various libraries SILC Toolkit provides, but
29 naturally they don't want the bloat of SILC protocol related stuff.
31 The Runtime library in SILC Toolkit is a general purpose runtime library,
32 like Glib and APR are. The runtime library is to be developed further
33 to provide alternative to Glib and APR.
35 The Crypto library in SILC Toolkit is a general purpose crypto library
36 providing pretty nice APIs compared to many other crypto libraries,
37 especially OpenSSL. The Crypto library is to be developed further
38 to include support for OpenPGP, X.509 and SSH2.
44 o SILC_PACKET_FLAG_ACK support. Implement ACK packet and packet payload
47 o All payload encoding routines should take SilcStack as argument.
49 o Remove SilcCommandCb from silccommand.h.
51 o All payload test routines into lib/silccore/tests/.
54 lib/silcclient, The Client Library
55 ==================================
57 o UDP SILC connection support to SILC server
59 o Giving WHOIS for nick that doesn't exist should remove any same
60 named entries from the client cache.
62 o peer-to-peer private messages
64 o Private message key request notification to application. See XXX in
67 o in JOIN notify handle resolving that timedout. Currently the user is
68 never joined the channel if this happens. What to do if message is
69 received from user that hasn't been resolved/joined?
71 o Add the SilcStream (socket stream) from the SilcPacketStream and
72 SilcSocket from the socket stream to SilcClientConnection for easier
73 access to them for programmers. Currently these have to be digged up
74 from the packet stream.
76 o Connection option that attemps to connect to remot host with various
77 different mechanisms: UDP 706, TCP 706, TCP 80, TCP 443, UDP 7706 and
78 TCP 7706. This is the so called hole punching mechanism.
80 o Message ACKing support.
82 o in /cmode and /cumode with +r, maybe the public key and private key
83 could be just some "string", which would then match to "string.pub" and
86 o If the SILC Events (see below) are implemented, perhaps client library
87 should provide events so that application developer has a choice of
88 developing the SILC app with callbacks or with events.
90 o Ability to recover from rekey errors, at least try to.
93 Runtime library, lib/silcutil/
94 ==============================
96 o Fix universal time decoding (doesn't accept all formats) in silctime.c.
98 o Add functions to manipulate environment variables. (***DONE)
100 o Add functions to loading shared/dynamic object symbols (replaces the
101 SIM library (lib/silcsim) and introduces generic library). Add this
102 to lib/silcutil/silcdll.[ch]. (***TESTING NEEDED WIN32, TODO Symbian)
104 o Add directory opening/traversing functions
106 o silc_getopt routines
108 o Add silc_stream_get_root and add get_root stream operation. It
109 returns the root of the stream or NULL if stream doesn't have root.
111 o Change some stream routines (like socket stream API) to accept ANY
112 stream and use silc_stream_get_root to get the socket stream from the
113 given stream. This will make various stream APIs more easier to use
114 when user doesn't have to dig up the correct stream.
116 o The SILC Event signals. Asynchronous events that can be created,
117 connected to and signalled. Either own event routines or glued into
120 SilcTask silc_schedule_task_add_event(SilcSchedule schedule,
121 const char *event, ...);
122 SilcBool silc_schedule_event_connect(SilcSchedule schedule,
124 SilcTaskCallback event_callback,
126 SilcBool silc_schedule_event_signal(SilcSchedule schedule,
127 const char *event, ...);
130 silc_schedule_task_add_event(schedule, "connected",
134 silc_schedule_event_connect(schedule, "connected", connected_cb, ctx);
135 silc_schedule_event_signal(schedule, "connected", integer, buf,
137 SILC_TASK_CALLBACK(connected_cb)
139 FooCtx ctx = context;
144 va_start(args, context);
145 integer = va_arg(args, SilcUInt32);
146 buf = va_arg(args, SilcBuffer);
151 Problems: Events would be SilcSchedule specific, and would not work on
152 multi-thread/multi-scheduler system. The events should be copyable
153 between schedulers. Another problem is the signal delivery. Do we
154 deliver them synchronously possibly from any thread to any other thread
155 or do we deliver them through the target schedulers. If we use the
156 schedulers then signalling would be asynchronous (data must be
157 duplicated and later freed) which is not very nice.
159 o If the event signals are added, the SILC_PARAM_* stuff needs to be
160 moved from silcbuffmt.h to silctypes.h or something similar.
162 o In case the SILC Events are done we shall create a new concept of
163 parent and child SilcSchedule's. When new SilcSchedule is created a
164 parent can be associated to it. This association could be done either
165 directly by the parent or by any other children. This way the signals
166 would in effect be global and would reach all children schedulers.
168 This relationship would be associative only. The schedulers are still
169 independent and run independently from each other. All schedulers
170 would be linked and could be accessed from any of the schedulers.
171 It should be possible to retrieve the parent and enumerate all children
172 from any of the schedulers.
174 SilcSchedule silc_schedule_init(int max_tasks, void *app_context,
175 SilcSchedule parent);
176 SilcSchedule silc_schedule_get_parent(SilcSchedule schedule);
178 o Additional scheduler changes: optimize silc_schedule_wakeup. Wakeup
179 only if the scheduler is actually waiting something. If it is
180 delivering tasks wakeup is not needed.
182 o Structured log messages to Log API. Allows machine readable log
183 messages. Would allow sending of any kind of data in a log message.
185 o Base64 to an own API (***DONE)
187 o Timer API (***DONE)
189 o Add builtin SOCKS and HTTP Proxy support, well the SOCKS at least.
190 SILC currently supports SOCKS4 and SOCKS5 but it needs to be compiled
193 o silc_stringprep to non-allocating version.
195 o silc_hash_table_replace -> silc_hash_table_set. Retain support for
196 silc_hash_table_replace as macro. (***DONE)
198 o SilcStack aware SilcHashTable. (***DONE)
200 o SilcStack aware SilcDList. (***DONE)
202 o Thread pool API. Add this to lib/silcutil/silcthread.[ch]. (***DONE)
204 o Fast mutex implementation. Fast rwlock implementation. Mutex and
205 rwlock implementation using atomic operations.
207 o Compression routines are missing. The protocol supports packet
208 compression thus it must be implemented. SILC Zip API must be
211 o Add new functions to SilcStack API in lib/silcutil/silcstack.[ch]. Add
212 silc_stack_[set|get]_alignment. It defines the default alignment used
213 when allocating memory from stack. It can be used to specify special
214 alignments too when needed (such as for hardware devices like crypto
215 accelerators). Move also the low level silc_stack_malloc and
216 silc_stack_realloc from silcstack_i.h to silcstack.h. Remove the
217 _ua unaligned memory allocation routines. Remove unaligned memory
218 allocation possibility. (***DONE)
220 o silc_stack_alloc shouldn't require multiple by 8 size argument, it
221 should figure it out itself.
223 o silc_malloc et. al. to respect --with-alignment.
225 o Add '%@' format to silc_snprintf functions. It marks for external
226 rendering function of following type:
228 /* Snprintf rendering function. The `data' is rendered into a string
229 and allocated string is returned. If NULL is returned the
230 rendering is skipped and ignored. If the returned string does
231 not fit to the destination buffer it may be truncated. */
232 typedef char *(*SilcSnprintfRender)(void *data);
234 It can work like following:
236 char *id_renderer(void *data)
239 id_to_str(tmp, sizeof(tmp), (SilcID *)data);
243 silc_snprintf(buf, sizeof(buf), "Client ID %@", id_renderer, client_id);
246 o Change silc_gettimeofday on Unix to use clock_gettime with REALTIME
247 clock if it is available, otherwise use gettimeofday(). (***DONE)
249 (o SilcIpAddr abstraction. Ipv4 and Ipv6 support to the abstaction.)
252 (o Generic SilcStatus or SilcResult that includes all possible status and
253 error conditions, including those of SILC protocol. Though, the SILC
254 protocol related status (currently in silcstatus.h) cannot be in
255 runtime library) maybe
257 (o SILC specific socket creation/closing routines to silcnet.h, wrappers
258 to all send(), recv(), sendto() etc. Bad thing is that we'd have to
259 define all socket options, sockaddrs, etc.) maybe
264 lib/silcutil/symbian/
265 =====================
267 o Something needs to be thought to the logging globals as well,
268 like silc_debug etc. They won't work on EPOC. Perhaps logging
269 and debugging is to be disabled on EPOC. The logging currently works
270 by it cannot be controlled, same with debugging.
273 SFTP Library, lib/silcsftp/
274 ===========================
276 o Read prefetch (read-ahead, reading ahead of time). Maybe if this can
280 SKR Library, lib/silcskr/
281 =========================
283 o Add fingerprint as search constraint.
285 o Add OpenPGP support. Adding, removing, fetching PGP keys. (Keyring
288 o Add support for importing public keys from a directory and/or from a
289 file. Add support for exporting the repository (different formats for
290 different key types?).
292 o Change the entire silc_skr_find API. Remove SilcSKRFind and just simply
293 add the find constraints as variable argument list to silc_skr_find, eg:
295 silc_skr_find(skr, schedule, callback, context,
296 SILC_SKR_FIND_PUBLIC_KEY, public_key,
297 SILC_SKR_FIND_COUNTRY, "FI",
298 SILC_SKR_FIND_USAGE, SILC_SKR_USAGE_AUTH,
301 NULL argument would be ignored and skipped.
303 o Add OR logical rule in addition of the current default AND, eg:
305 // Found key(s) MUST have this public key AND this country.
306 silc_skr_find(skr, schedule, callback, context,
307 SILC_SKR_FIND_RULE_AND,
308 SILC_SKR_FIND_PUBLIC_KEY, public_key,
309 SILC_SKR_FIND_COUNTRY, "FI",
312 // Found key(s) MUST have this public key OR this key context
313 silc_skr_find(skr, schedule, callback, context,
314 SILC_SKR_FIND_RULE_OR,
315 SILC_SKR_FIND_PUBLIC_KEY, public_key,
316 SILC_SKR_FIND_CONTEXT, key_context,
319 o SilcStack to SKR API.
322 Crypto Library, lib/silccrypt/
323 ==============================
325 o Add silc_crypto_init and silc_crypto_uninit. The _init should take
326 SilcStack that will act as global memory pool for all of crypto
327 library. It should not be necessary anymore to separately register
328 default ciphers, HMACs, etc, the _init would do that. However, if
329 user after _init calls silc_pkcs_register, for example, it would take
330 preference over the default once, ie. user can always dictate the
331 order of algorithms. (***DONE)
333 o Add fingerprint to SilcSILCPublicKey and retrieval to silcpk.h, and
334 possibly to silcpkcs.h.
336 /* Return fingerprint of the `public_key'. Returns also the algorithm
337 that has been used to make the fingerprint. */
338 const unsigned char *
339 silc_pkcs_get_fingerprint(SilcPublicKey public_key,
340 const char **hash_algorithm,
341 SilcUInt32 *fingerprint_len);
343 o Change SILC PKCS API to asynchronous, so that accelerators can be used.
344 All PKCS routines should now take callbacks as argument and they should
345 be delivered to SilcPKCSObject and SilcPKCSAlgorithm too. (***DONE)
347 o Change PKCS Algorithm API to take SilcPKCSAlgorithm as argument to
348 encrypt, decrypt, sign and verify functions. We may need to for exmaple
349 check the alg->hash, supported hash functions. Maybe deliver it also
350 to all other functions in SilcPKCSAlgorithm to be consistent. (***DONE)
352 o Add DSA support to SILC public key.
354 o Add DSS support. (***DONE)
356 o Implement the defined SilcDH API. The definition is in
357 lib/silccrypt/silcdh.h. Make sure it is asynchronous so that it can
358 be accelerated. Also take into account that it could use elliptic
365 o AES CBC is missing proper alignment code (see silc_1_1_branch).
367 o All cipher, hash, hmac etc. allocation routines should take their name
368 in as const char * not const unsigned char *. (***DONE)
371 SILC Accelerator Library
372 ========================
374 o SILC Accelerator API. Provides generic way to use different kind of
375 accelerators. Basically implements SILC PKCS API so that SilcPublicKey
376 and SilcPrivateKey can be used but they call the accelerators.
379 o Implement software accelerator. It is a thread pool system where the
380 public key and private key operations are executed in threads.
383 o Add init options to SilcAcceleratorObject as a SilcAcceleratorOption
384 structure. Each accelerator defines the options that they support and
385 can be retrieved from the SilcAccelerator with silc_acc_get_options.
386 The format must also be machine parseable. The structure can be of the
389 typedef struct SilcAcceleratorOptionStruct {
390 const char *option; /* Option name */
391 const char *display_name; /* Option displayable name */
392 SilcParamType type; /* Option data format */
393 } *SilcAcceleratorOption;
395 For software accelerator it could be for example:
397 { "min_threads", "Minimum threads", SILC_PARAM_UINT32 },
398 { "max_threads", "Maximum threads", SILC_PARAM_UINT32 },
400 The accelerator itself doesn't have to use the option structure to
401 parse the options if not wanted. It is defined for the caller so
402 they can learn the supported options in a well defined way.
404 o Diffie-Hellman acceleration
406 (o Symmetric key cryptosystem acceleration? They are always sycnhronouos
407 even with hardware acceleration so the crypto API shouldn't require
414 o Import TFM. We want TFM's speed but its memory requirements are
415 just too much. By default it uses large pre-allocated tables which
416 will eat memory when there are thousands of public keys in system.
417 We probably want to change TFM's fp_int dynamic so that a specific
418 size can be allocated for the int. We could have two new functions:
420 SilcBool silc_mp_init_size(SilcMPInt *mp, SilcUInt32 bit_size);
421 SilcBool silc_mp_sinit_size(SilcStack stack, SilcMPInt *mp,
422 SilcUInt32 bit_size);
424 Which by default allocates `bit_size' bits instead of some default
425 value. silc_mp_init would allocate the default FP_SIZE with TFM
426 and do normal init with TMA and GMP. _init_size with TMA and GMP
427 would be same as _init.
429 o Add AND, OR and XOR support to TFM or ask Tom to do it.
431 o The SILC MP API function must start returning indication of success
432 and failure of the operation.
434 o Do SilcStack support for silc_mp_init, silc_mp_init_size and other
435 any other MP function (including utility ones) that may allocate
438 o Prime generation progress using callback instead of printing to
441 o All utility functions should be made non-allocating ones.
444 SILC XML Library, lib/silcxml/
445 ==============================
447 o SILC XML API (wrapper to expat). Look at the expat API and simplify
448 it. The SILC XML API should have at most 8-10 API functions. It should
449 be possible to create full XML parser with only one function. And, it
450 should be possible to have a function that is able to parse an entire
451 XML document. It should also have a parser function to be able to
452 parse a stream of XML data (SilcStream). It MUST NOT have operations
453 that require multiple function calls to be able to execute that one
454 operation (like creating parser).
457 lib/silcske/silcske.[ch]
458 ========================
460 o Ratelimit to UDP/IP transport for incoming packets.
466 o Negative integer encoding is missing, add it.
468 o SILC_ASN1_CHOICE should perhaps return an index what choice in the
469 choice list was found. Currently it is left for caller to figure out
470 which choice was found. (***DONE)
472 o SILC_ASN1_NULL in decoding should return SilcBool whether or not
473 the NULL was present. It's important when it's SILC_ASN1_OPTIONAL
474 and we need to know whether it was present or not. (***DONE)
480 o OpenPGP certificate support, allowing the use of PGP public keys.
486 o SSH2 public key/private key support, allowing the use of SSH2 keys.
493 o PKIX implementation
499 o Deprecate the old server. Write interface for the new lib/silcserver
500 server library. The interface should work on Unix/Linux systems.
502 o Consider deprecating also the old config file format and use XML
503 istead. This should require SILC XML API implementation first.
505 o The configuration must support dynamic router and server connections.
506 The silcd must work without specifying any servers or routers to
509 o The configuration must support specifying whether the server is
510 SILC Server or SILC Router. This should not be deduced from the
511 configuration as it was in < 1.2.
513 o The configuration must support specifying the ciphers and hmacs and
514 their order so that user can specify which algorithms take preference.
520 o Rewrite the entire server. Deprecate apps/silcd as the main server
521 implementation and create lib/silcserver/. It is a platform
522 independent server library. The apps/silcd will merely provide a
523 a simple interface for the library.
525 o Write the SILC Server library extensively using SILC FSM.
527 o Server library must support multiple networks. This means that one
528 server must be able to create multiple connections that each reach
529 different SILC network. This means also that all cache's etc. must
530 be either connection-specific or network-specific.
532 o Library must support dynamic router and server connections. This means
533 that connections are create only when they are needed, like when someone
534 says JOIN foo@foo.bar.com or WHOIS foobar@silcnet.org.
536 o Library must support server-to-server connections even though protocol
537 prohibits that. The responder of the connection should automatically
538 act as a router. The two servers create an own, isolated, SILC network.
539 To be used specifically with dynamic connections.
541 o Library must support multiple threads and must be entirely thread safe.
543 o Library must have support for SERVICE command.
545 o Both UDP and TCP support for incoming connecetions. Maintaining long
548 o The server must be able to run behind NAT device. This means that
549 Server ID must be based on public IP instead of private IP (See
550 also NAT detection protocol in SILC protocol specification).
552 o The following data must be in per-connection context: client id cache,
553 server id cache, channel id cache, all statistics must be
556 o The following data must be in per-thread context: command context
557 freelist/pool, pending commands, random number generator.
559 o Do inccoming packet processing in an own FSM thread in the
560 server-threads FSM. Same as in client library.
562 o Binding to other ports than 706 too. To allow easier traversing
563 through NATs and firewalls server should also bind to 80, 443 and 7706
564 by default (at least try to bind). Connections must work normally
565 even if they were established to some other port other than 706.
567 Connection option that attemps to connect to remot server with various
568 different mechanisms: UDP 706, TCP 706, TCP 80, TCP 443, UDP 7706 and
569 TCP 7706. This is the so called hole punching mechanism.
571 o Ability to recover from rekey errors, at least try to.
573 o Reference count all Silc*Entry structures.
575 Some issues that must be kept in mind from 1.0 and 1.1 silcd's:
577 o The server and router software MUST work out of the box. After
578 installation the server must not require any configuration to run the
579 most basic working configuration. No defining IP addresses, etc.
580 The server must work just by running it.
582 o The SERVER_SIGNOFF notify handing is not optimal, because it'll
583 cause sending of multiple SIGNOFF notify's instead of the one
584 SERVER_SIGNOFF notify that the server received. This should be
585 optimized so that the only SERVER_SIGNOFF is sent and not
586 SIGNOFF of notify at all (using SIGNOFF takes the idea about
587 SERVER_SIGNOFF away entirely).
589 o Another SERVER_SIGNOFF opt/bugfix: Currently the signoff is
590 sent to a client if it is on same channel as the client that
591 signoffed. However, the entire SERVER_SIGNOFF list is sent to
592 the client, ie. it may receive clients that was not on the
593 same channel. This is actually against the specs. It must be
594 done per channel. It shouldn't receive the whole list just
595 because one client happened to be on same channel.
597 o If client's public key is saved in the server (and doing public key
598 authentication) then the hostname and the username information could
599 be taken from the public key. Should be a configuration option!
601 o Add a timeout to handling incoming JOIN commands. It should be
602 enforced that JOIN command is executed only once in a second or two
603 seconds. Now it is possible to accept n incoming JOIN commands
604 and process them without any timeouts. THis must be employed because
605 each JOIN command will create and distribute the new channel key
606 to everybody on the channel.
608 o Related to above. If multiple JOINs are received in sequence perhaps
609 new key should be created only once, if the JOINs are handeled at the same
610 time. Now we create multiple keys and never end up using them because
611 many JOINs are processed at the same time in sequence. Only the last
612 key ends up being used.