X-Git-Url: http://git.silcnet.org/gitweb/?p=crypto.git;a=blobdiff_plain;f=TODO;h=5bfd5f6937d9719e6af1070d12e0d843c8b0d045;hp=eb380fad03f905c5998678592a8a1dbaf249025a;hb=79284c6f2e3df2de326793b1f9ff038ad88f9e0f;hpb=7431bd61b7149a1b25ccd969cfbb05316ff9fa59 diff --git a/TODO b/TODO index eb380fad..5bfd5f69 100644 --- a/TODO +++ b/TODO @@ -1,65 +1,608 @@ -TODO/bugs in Irssi SILC client +TODO for 1.2 And Beyond +======================= + +NOTE: Any item that doesn't have (***DONE) in it, isn't done yet. The +(***TESTING NEEDED) means that the item has been done but not yet properly +tested. + +NOTE: A TODO entry does not mean that it is ever going to be done. Some +of the entries may be just ideas, good, bad or ugly. If you want to work +on some of the TODO entries simply let us know about it by dropping a note +to silc-devel mailing list or appear on 'silc' channel on SILCNet. + + +General +======= + + o Create apps/tutorial containing various Toolkit API tutorials. + + o The Toolkit split. The Toolkit is to be splitted in parts. How many + parts and what the parts are isn't decided yet. Each part is a separate + software package. Current thinking is of the following: + + SILC Toolkit SILC protocol, client and server library + SILC Runtime Toolkit runtime library + SILC Crypto Toolkit crypto, asn1, math, skr, pgp, etc. + + The rationale for this is of course that other than SILC projects + might like to use the various libraries SILC Toolkit provides, but + naturally they don't want the bloat of SILC protocol related stuff. + + The Runtime library in SILC Toolkit is a general purpose runtime library, + like Glib and APR are. The runtime library is to be developed further + to provide alternative to Glib and APR. + + The Crypto library in SILC Toolkit is a general purpose crypto library + providing pretty nice APIs compared to many other crypto libraries, + especially OpenSSL. The Crypto library is to be developed further + to include support for OpenPGP, X.509 and SSH2. + + +lib/silccore +============ + + o SILC_PACKET_FLAG_ACK support. Implement ACK packet and packet payload + to silcpacket.c. + + o All payload encoding routines should take SilcStack as argument. + + o Remove SilcCommandCb from silccommand.h. + + o All payload test routines into lib/silccore/tests/. + + +lib/silcclient, The Client Library +================================== + + o UDP SILC connection support to SILC server + + o Giving WHOIS for nick that doesn't exist should remove any same + named entries from the client cache. + + o peer-to-peer private messages + + o Private message key request notification to application. See XXX in + client_prvmsg.c. + + o in JOIN notify handle resolving that timedout. Currently the user is + never joined the channel if this happens. What to do if message is + received from user that hasn't been resolved/joined? + + o Add the SilcStream (socket stream) from the SilcPacketStream and + SilcSocket from the socket stream to SilcClientConnection for easier + access to them for programmers. Currently these have to be digged up + from the packet stream. + + o Connection option that attemps to connect to remot host with various + different mechanisms: UDP 706, TCP 706, TCP 80, TCP 443, UDP 7706 and + TCP 7706. This is the so called hole punching mechanism. + + o Message ACKing support. + + o in /cmode and /cumode with +r, maybe the public key and private key + could be just some "string", which would then match to "string.pub" and + "string.prv". + + o If the SILC Events (see below) are implemented, perhaps client library + should provide events so that application developer has a choice of + developing the SILC app with callbacks or with events. + + o Ability to recover from rekey errors, at least try to. + + +Runtime library, lib/silcutil/ ============================== - o UTF-8 encode/decode WHOIS userinfos, topic, etc. + o Fix universal time decoding (doesn't accept all formats) in silctime.c. - o Manaul file for silc(1) and silc.conf. + o Add functions to manipulate environment variables. (***DONE) - o Testing + o Add functions to loading shared/dynamic object symbols (replaces the + SIM library (lib/silcsim) and introduces generic library). Add this + to lib/silcutil/silcdll.[ch]. (***TESTING NEEDED WIN32, TODO Symbian) + o Add directory opening/traversing functions -TODO/bugs In SILC Client Library -================================ + o silc_getopt routines - o Testing + o Add silc_stream_get_root and add get_root stream operation. It + returns the root of the stream or NULL if stream doesn't have root. + o Change some stream routines (like socket stream API) to accept ANY + stream and use silc_stream_get_root to get the socket stream from the + given stream. This will make various stream APIs more easier to use + when user doesn't have to dig up the correct stream. -TODO/bugs In SILC Server -======================== + o The SILC Event signals. Asynchronous events that can be created, + connected to and signalled. Either own event routines or glued into + SilcSchedule: - o If client's public key is saved in the server (and doing public key - authentication) then the hostname and the username information could - be taken from the public key. Should be a configuration option! + SilcTask silc_schedule_task_add_event(SilcSchedule schedule, + const char *event, ...); + SilcBool silc_schedule_event_connect(SilcSchedule schedule, + const char *event, + SilcTaskCallback event_callback, + void *context); + SilcBool silc_schedule_event_signal(SilcSchedule schedule, + const char *event, ...); + + Example: + silc_schedule_task_add_event(schedule, "connected", + SILC_PARAM_UI32_INT, + SILC_PARAM_BUFFER, + SILC_PARAM_END); + silc_schedule_event_connect(schedule, "connected", connected_cb, ctx); + silc_schedule_event_signal(schedule, "connected", integer, buf, + SILC_PARAM_END); + SILC_TASK_CALLBACK(connected_cb) + { + FooCtx ctx = context; + va_list args; + SilcUInt32 integer; + SilcBuffer buf; + + va_start(args, context); + integer = va_arg(args, SilcUInt32); + buf = va_arg(args, SilcBuffer); + va_end(args); + ... + } + + Problems: Events would be SilcSchedule specific, and would not work on + multi-thread/multi-scheduler system. The events should be copyable + between schedulers. Another problem is the signal delivery. Do we + deliver them synchronously possibly from any thread to any other thread + or do we deliver them through the target schedulers. If we use the + schedulers then signalling would be asynchronous (data must be + duplicated and later freed) which is not very nice. + + o If the event signals are added, the SILC_PARAM_* stuff needs to be + moved from silcbuffmt.h to silctypes.h or something similar. + + o In case the SILC Events are done we shall create a new concept of + parent and child SilcSchedule's. When new SilcSchedule is created a + parent can be associated to it. This association could be done either + directly by the parent or by any other children. This way the signals + would in effect be global and would reach all children schedulers. + + This relationship would be associative only. The schedulers are still + independent and run independently from each other. All schedulers + would be linked and could be accessed from any of the schedulers. + It should be possible to retrieve the parent and enumerate all children + from any of the schedulers. - o Backup router related issues: + SilcSchedule silc_schedule_init(int max_tasks, void *app_context, + SilcSchedule parent); + SilcSchedule silc_schedule_get_parent(SilcSchedule schedule); - o Add special handling in router and server for "connection - timed out" error. Be optimistic. + o Additional scheduler changes: optimize silc_schedule_wakeup. Wakeup + only if the scheduler is actually waiting something. If it is + delivering tasks wakeup is not needed. - o Manual file for silcd(8) and silcd.conf. + o Structured log messages to Log API. Allows machine readable log + messages. Would allow sending of any kind of data in a log message. - o Testing + o Base64 to an own API (***DONE) + o Timer API (***DONE) -TODO/bugs In SILC Libraries + o Add builtin SOCKS and HTTP Proxy support, well the SOCKS at least. + SILC currently supports SOCKS4 and SOCKS5 but it needs to be compiled + in separately. + + o silc_stringprep to non-allocating version. + + o silc_hash_table_replace -> silc_hash_table_set. Retain support for + silc_hash_table_replace as macro. (***DONE) + + o SilcStack aware SilcHashTable. (***DONE) + + o SilcStack aware SilcDList. (***DONE) + + o Thread pool API. Add this to lib/silcutil/silcthread.[ch]. (***DONE) + + o Fast mutex implementation. Fast rwlock implementation. Mutex and + rwlock implementation using atomic operations. + + o Compression routines are missing. The protocol supports packet + compression thus it must be implemented. SILC Zip API must be + defined. + + o Add new functions to SilcStack API in lib/silcutil/silcstack.[ch]. Add + silc_stack_[set|get]_alignment. It defines the default alignment used + when allocating memory from stack. It can be used to specify special + alignments too when needed (such as for hardware devices like crypto + accelerators). Move also the low level silc_stack_malloc and + silc_stack_realloc from silcstack_i.h to silcstack.h. Remove the + _ua unaligned memory allocation routines. Remove unaligned memory + allocation possibility. (***DONE) + + o silc_stack_alloc shouldn't require multiple by 8 size argument, it + should figure it out itself. + + o silc_malloc et. al. to respect --with-alignment. + + o Add '%@' format to silc_snprintf functions. It marks for external + rendering function of following type: + + /* Snprintf rendering function. The `data' is rendered into a string + and allocated string is returned. If NULL is returned the + rendering is skipped and ignored. If the returned string does + not fit to the destination buffer it may be truncated. */ + typedef char *(*SilcSnprintfRender)(void *data); + + It can work like following: + + char *id_renderer(void *data) + { + char tmp[32]; + id_to_str(tmp, sizeof(tmp), (SilcID *)data); + return strdup(tmp); + } + + silc_snprintf(buf, sizeof(buf), "Client ID %@", id_renderer, client_id); + (***DONE) + + o Change silc_gettimeofday on Unix to use clock_gettime with REALTIME + clock if it is available, otherwise use gettimeofday(). (***DONE) + + (o SilcIpAddr abstraction. Ipv4 and Ipv6 support to the abstaction.) + maybe + + (o Generic SilcStatus or SilcResult that includes all possible status and + error conditions, including those of SILC protocol. Though, the SILC + protocol related status (currently in silcstatus.h) cannot be in + runtime library) maybe + + (o SILC specific socket creation/closing routines to silcnet.h, wrappers + to all send(), recv(), sendto() etc. Bad thing is that we'd have to + define all socket options, sockaddrs, etc.) maybe + + (o mmap) maybe + + +lib/silcutil/symbian/ +===================== + + o Something needs to be thought to the logging globals as well, + like silc_debug etc. They won't work on EPOC. Perhaps logging + and debugging is to be disabled on EPOC. The logging currently works + by it cannot be controlled, same with debugging. + + +SFTP Library, lib/silcsftp/ =========================== - o WIN32 silc_net_create_connection_async does not work the same way - than on Unix. Do it with threads on WIN32. The function works but - is not actually async currently (Fix this to 0.9.x). + o Read prefetch (read-ahead, reading ahead of time). Maybe if this can + be done easily. + + +SKR Library, lib/silcskr/ +========================= + + o Add fingerprint as search constraint. + + o Add OpenPGP support. Adding, removing, fetching PGP keys. (Keyring + support?) + + o Add support for importing public keys from a directory and/or from a + file. Add support for exporting the repository (different formats for + different key types?). + + o Change the entire silc_skr_find API. Remove SilcSKRFind and just simply + add the find constraints as variable argument list to silc_skr_find, eg: + + silc_skr_find(skr, schedule, callback, context, + SILC_SKR_FIND_PUBLIC_KEY, public_key, + SILC_SKR_FIND_COUNTRY, "FI", + SILC_SKR_FIND_USAGE, SILC_SKR_USAGE_AUTH, + SILC_SKR_FIND_END); + + NULL argument would be ignored and skipped. + + o Add OR logical rule in addition of the current default AND, eg: + + // Found key(s) MUST have this public key AND this country. + silc_skr_find(skr, schedule, callback, context, + SILC_SKR_FIND_RULE_AND, + SILC_SKR_FIND_PUBLIC_KEY, public_key, + SILC_SKR_FIND_COUNTRY, "FI", + SILC_SKR_FIND_END); + + // Found key(s) MUST have this public key OR this key context + silc_skr_find(skr, schedule, callback, context, + SILC_SKR_FIND_RULE_OR, + SILC_SKR_FIND_PUBLIC_KEY, public_key, + SILC_SKR_FIND_CONTEXT, key_context, + SILC_SKR_FIND_END); + + o SilcStack to SKR API. + + +Crypto Library, lib/silccrypt/ +============================== + + o Add silc_crypto_init and silc_crypto_uninit. The _init should take + SilcStack that will act as global memory pool for all of crypto + library. It should not be necessary anymore to separately register + default ciphers, HMACs, etc, the _init would do that. However, if + user after _init calls silc_pkcs_register, for example, it would take + preference over the default once, ie. user can always dictate the + order of algorithms. (***DONE) + + o Add fingerprint to SilcSILCPublicKey and retrieval to silcpk.h, and + possibly to silcpkcs.h. + + /* Return fingerprint of the `public_key'. Returns also the algorithm + that has been used to make the fingerprint. */ + const unsigned char * + silc_pkcs_get_fingerprint(SilcPublicKey public_key, + const char **hash_algorithm, + SilcUInt32 *fingerprint_len); + + o Change SILC PKCS API to asynchronous, so that accelerators can be used. + All PKCS routines should now take callbacks as argument and they should + be delivered to SilcPKCSObject and SilcPKCSAlgorithm too. (***DONE) + + o Change PKCS Algorithm API to take SilcPKCSAlgorithm as argument to + encrypt, decrypt, sign and verify functions. We may need to for exmaple + check the alg->hash, supported hash functions. Maybe deliver it also + to all other functions in SilcPKCSAlgorithm to be consistent. (***DONE) + + o Add DSA support to SILC public key. + + o Add DSS support. (***DONE) + + o Implement the defined SilcDH API. The definition is in + lib/silccrypt/silcdh.h. Make sure it is asynchronous so that it can + be accelerated. Also take into account that it could use elliptic + curves. + + o Add ECDSA support. + + o Add ECDH support. + + o AES CBC is missing proper alignment code (see silc_1_1_branch). + + o All cipher, hash, hmac etc. allocation routines should take their name + in as const char * not const unsigned char *. (***DONE) + + +SILC Accelerator Library +======================== + + o SILC Accelerator API. Provides generic way to use different kind of + accelerators. Basically implements SILC PKCS API so that SilcPublicKey + and SilcPrivateKey can be used but they call the accelerators. + (***DONE) + + o Implement software accelerator. It is a thread pool system where the + public key and private key operations are executed in threads. + (***DONE) + + o Add init options to SilcAcceleratorObject as a SilcAcceleratorOption + structure. Each accelerator defines the options that they support and + can be retrieved from the SilcAccelerator with silc_acc_get_options. + The format must also be machine parseable. The structure can be of the + following format: + + typedef struct SilcAcceleratorOptionStruct { + const char *option; /* Option name */ + const char *display_name; /* Option displayable name */ + SilcParamType type; /* Option data format */ + } *SilcAcceleratorOption; + + For software accelerator it could be for example: + + { "min_threads", "Minimum threads", SILC_PARAM_UINT32 }, + { "max_threads", "Maximum threads", SILC_PARAM_UINT32 }, + + The accelerator itself doesn't have to use the option structure to + parse the options if not wanted. It is defined for the caller so + they can learn the supported options in a well defined way. + + o Diffie-Hellman acceleration + + (o Symmetric key cryptosystem acceleration? They are always sycnhronouos + even with hardware acceleration so the crypto API shouldn't require + changes.) maybe + + +lib/silcmath +============ + + o Import TFM. We want TFM's speed but its memory requirements are + just too much. By default it uses large pre-allocated tables which + will eat memory when there are thousands of public keys in system. + We probably want to change TFM's fp_int dynamic so that a specific + size can be allocated for the int. We could have two new functions: + + SilcBool silc_mp_init_size(SilcMPInt *mp, SilcUInt32 bit_size); + SilcBool silc_mp_sinit_size(SilcStack stack, SilcMPInt *mp, + SilcUInt32 bit_size); + + Which by default allocates `bit_size' bits instead of some default + value. silc_mp_init would allocate the default FP_SIZE with TFM + and do normal init with TMA and GMP. _init_size with TMA and GMP + would be same as _init. + + o Add AND, OR and XOR support to TFM or ask Tom to do it. + + o The SILC MP API function must start returning indication of success + and failure of the operation. + + o Do SilcStack support for silc_mp_init, silc_mp_init_size and other + any other MP function (including utility ones) that may allocate + memory. + + o All utility functions should be made non-allocating ones. + + +SILC XML Library, lib/silcxml/ +============================== + + o SILC XML API (wrapper to expat). Look at the expat API and simplify + it. The SILC XML API should have at most 8-10 API functions. It should + be possible to create full XML parser with only one function. And, it + should be possible to have a function that is able to parse an entire + XML document. It should also have a parser function to be able to + parse a stream of XML data (SilcStream). It MUST NOT have operations + that require multiple function calls to be able to execute that one + operation (like creating parser). + + +lib/silcske/silcske.[ch] +======================== + + o Ratelimit to UDP/IP transport for incoming packets. -TODO in Toolkit Documentation -============================= +lib/silcasn1 +============ -Stuff that needs to be done in order to complete the Tooolkit Reference -Manual (Do these to 0.9.x). + o Negative integer encoding is missing, add it. - o ROBOdoc documenting missing from lib/silccrypt/silccipher.h. + o SILC_ASN1_CHOICE should perhaps return an index what choice in the + choice list was found. Currently it is left for caller to figure out + which choice was found. (***DONE) - o ROBOdoc documenting missing from lib/silccrypt/silcpkcs.h. + o SILC_ASN1_NULL in decoding should return SilcBool whether or not + the NULL was present. It's important when it's SILC_ASN1_OPTIONAL + and we need to know whether it was present or not. (***DONE) - o Write "Programming with Toolkit" document, describing how to build - Toolkit, how the build system works, where is everything, how - new (external) projects can be glued into Toolkit (use irssi as an - example), and how external projects can use Toolkit without gluing into - it (how to link etc), debugging, architecture, types, etc. - o Write "Platform Implementations" document to describe what platforms - Toolkit support, what has been implemented, what has not been, what - works differently etc. +lib/silcpgp +=========== + + o OpenPGP certificate support, allowing the use of PGP public keys. + + +lib/silcssh +=========== + + o SSH2 public key/private key support, allowing the use of SSH2 keys. + RFC 4716. (***DONE) + + +lib/silcpkix +============ + + o PKIX implementation + + +apps/silcd +========== + + o Deprecate the old server. Write interface for the new lib/silcserver + server library. The interface should work on Unix/Linux systems. + + o Consider deprecating also the old config file format and use XML + istead. This should require SILC XML API implementation first. + + o The configuration must support dynamic router and server connections. + The silcd must work without specifying any servers or routers to + connect to. + + o The configuration must support specifying whether the server is + SILC Server or SILC Router. This should not be deduced from the + configuration as it was in < 1.2. + + o The configuration must support specifying the ciphers and hmacs and + their order so that user can specify which algorithms take preference. + + +lib/silcserver +============== + + o Rewrite the entire server. Deprecate apps/silcd as the main server + implementation and create lib/silcserver/. It is a platform + independent server library. The apps/silcd will merely provide a + a simple interface for the library. + + o Write the SILC Server library extensively using SILC FSM. + + o Server library must support multiple networks. This means that one + server must be able to create multiple connections that each reach + different SILC network. This means also that all cache's etc. must + be either connection-specific or network-specific. + + o Library must support dynamic router and server connections. This means + that connections are create only when they are needed, like when someone + says JOIN foo@foo.bar.com or WHOIS foobar@silcnet.org. + + o Library must support server-to-server connections even though protocol + prohibits that. The responder of the connection should automatically + act as a router. The two servers create an own, isolated, SILC network. + To be used specifically with dynamic connections. + + o Library must support multiple threads and must be entirely thread safe. + + o Library must have support for SERVICE command. + + o Both UDP and TCP support for incoming connecetions. Maintaining long + term UDP sessions. + + o The server must be able to run behind NAT device. This means that + Server ID must be based on public IP instead of private IP. + + o The following data must be in per-connection context: client id cache, + server id cache, channel id cache, all statistics must be + per-connection. + + o The following data must be in per-thread context: command context + freelist/pool, pending commands, random number generator. + + o Do inccoming packet processing in an own FSM thread in the + server-threads FSM. Same as in client library. + + o Binding to other ports than 706 too. To allow easier traversing + through NATs and firewalls server should bind to 80, 443 and 7706 + by default (at least try to bind). Connections must work normally + even if they were established to some other port other than 706. + + Connection option that attemps to connect to remot server with various + different mechanisms: UDP 706, TCP 706, TCP 80, TCP 443, UDP 7706 and + TCP 7706. This is the so called hole punching mechanism. + + o Ability to recover from rekey errors, at least try to. + + o Reference count all Silc*Entry structures. + + Some issues that must be kept in mind from 1.0 and 1.1 silcd's: + + o The server and router software MUST work out of the box. After + installation the server must not require any configuration to run the + most basic working configuration. No defining IP addresses, etc. + The server must work just by running it. + + o The SERVER_SIGNOFF notify handing is not optimal, because it'll + cause sending of multiple SIGNOFF notify's instead of the one + SERVER_SIGNOFF notify that the server received. This should be + optimized so that the only SERVER_SIGNOFF is sent and not + SIGNOFF of notify at all (using SIGNOFF takes the idea about + SERVER_SIGNOFF away entirely). + + o Another SERVER_SIGNOFF opt/bugfix: Currently the signoff is + sent to a client if it is on same channel as the client that + signoffed. However, the entire SERVER_SIGNOFF list is sent to + the client, ie. it may receive clients that was not on the + same channel. This is actually against the specs. It must be + done per channel. It shouldn't receive the whole list just + because one client happened to be on same channel. + + o If client's public key is saved in the server (and doing public key + authentication) then the hostname and the username information could + be taken from the public key. Should be a configuration option! - o Automatically generated index (would be nice). + o Add a timeout to handling incoming JOIN commands. It should be + enforced that JOIN command is executed only once in a second or two + seconds. Now it is possible to accept n incoming JOIN commands + and process them without any timeouts. THis must be employed because + each JOIN command will create and distribute the new channel key + to everybody on the channel. - o Searching of predefined keywords, exact and partial matches (would be - nice). + o Related to above. If multiple JOINs are received in sequence perhaps + new key should be created only once, if the JOINs are handeled at the same + time. Now we create multiple keys and never end up using them because + many JOINs are processed at the same time in sequence. Only the last + key ends up being used.