Check function return value in libtls

Check function return value in libtls

EVP_EncryptInit_ex, EVP_DecryptInit_ex and HMAC_Init_ex are possible to
fail and return error.
Error from these functions will be fatal for the callback, and I choose to
return -1.
SSL_CTX_set_tlsext_ticket_key_cb.3 explains the return value of callback.

This also could fix Coverity CID 345319.

ok jsing@ tb@