SSL Server mit BOOST ASIO schafft nicht mehr wie ~1000 Verbindungen
-
Hallo Leute,
ich habe einen Server mit C++ und der BOOST ASIO Bibliothek geschrieben.
Der Server soll möglichst viele Verbindungen annehmen (läuft auf einer Debian Maschine) und als Gateway zwischen Client und Server fungieren, sprich pro verbundenem Client zwei offene Sockets. Das System hat nur ein Ehternet Interface.Nun ist es so dass ab einer bestimmten Anzahl von offenen Verbindungen das ganze System sehr zäh wird, CPU Auslastung aber trotzdem nie mehr wie wie 10% beträgt.
Meistens dümpelt die Auslastung zwischen 0% und 5%. Bei ca. 800 - 1000 verbundenen Clients (netstat -apn | grep 443.*ESTABLISHED | wc -l) nimmt der Server nur nach mehreren Sekunden neue Verbindungen an.
Auch der locale Verbindungsaufbau mittels openssl s_client -connect localhost:443 dauert über 10 Sekunden.
Laut nload habe ich einen Durschnittlichen Traffic von 500 Kilobyte/s rein und raus.
Ich habe schon sämtliche Werte im System hochgeschraubt, allerdings bin ich mir nicht sicher wie ich das Bottleneck sicher bestimmen kann.Wenn ich den SSL Server neustarte, klappt die Verbindung in einem Sekundenbruchteil. Unter /var/log/ konnte ich den Logfiles keine Fehler erkennen.
Hat jemand eine Idee ?
Folgende Änderungen habe ich an der /etc/sysctl.conf angehängt, welche schon eine Besserung gebracht haben.
# Use the full range of ports. net.ipv4.ip_local_port_range = 1024 65535 # Maximum number of remembered connection requests, which are still did not receive an acknowledgment from connecting client. The default value is 1024 for systems with more than 128Mb of memory net.ipv4.tcp_max_syn_backlog = 8192 # Limit of socket listen() backlog, known in userspace as SOMAXCONN, default is 128 net.core.somaxconn=8192 # max receive buffer for all type of connection net.core.rmem_max=16777216 # max send buffer for all type of connection net.core.wmem_max=16777216 # default receive buffer size for all type of connection net.core.rmem_default=65536 # default send buffer size for all type of connection net.core.wmem_default=65536 #The tcp_mem variable defines how the TCP stack should behave when it comes to memory usage. ... #The first value specified in the tcp_mem variable tells the kernel the low threshold. Below this point, the TCP stack do not bother at all about putting any pressure on the memory usage by different TCP sockets. ... #The second value tells the kernel at which point to start pressuring memory usage down. ... #The final value tells the kernel how many memory pages it may use maximally. If this value is reached, TCP streams and packets start getting dropped until we reach a lower memory usage again. This value includes all TCP sockets currently in use." net.ipv4.tcp_mem='16777216 16777216 16777216' #The first value tells the kernel the minimum receive buffer for each TCP connection, and this buffer is always allocated to a TCP socket, even under high pressure on the system. ... #The second value specified tells the kernel the default receive buffer allocated for each TCP socket. This value overrides the /proc/sys/net/core/rmem_default value used by other protocols. ... #The third and last value specified in this variable specifies the maximum receive buffer that can be allocated for a TCP socket." net.ipv4.tcp_rmem='4096 65536 16777216' #This variable takes 3 different values which holds information on how much TCP sendbuffer memory space each TCP socket has to use. Every TCP socket has this much buffer space to use before the buffer is filled up. Each of the three values are used under different conditions. ... #The first value in this variable tells the minimum TCP send buffer space available for a single TCP socket. ... #The second value in the variable tells us the default buffer space allowed for a single TCP socket to use. ... #The third value tells the kernel the maximum TCP send buffer space." net.ipv4.tcp_wmem='4096 65536 16777216'Hier die Limits für den User unter dem der SSL Server läuft.
Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size 0 unlimited bytes Max resident set unlimited unlimited bytes Max processes 15794 15794 processes Max open files 65536 65536 files Max locked memory 65536 65536 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 15794 15794 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited usGruß Till
-
Hier noch ein paar mehr Infos, nachdem ich die Frage auch im Debian Forum gepostet habe, habe ich havedged installiert, damit dem System mehr entropie zur Verfügung steht.
Ich muss sagen dass ich mich zum ersten mal mit dem Thema Systemoptimierung beschäftige,
da ich bisher kein Programm geschrieben habe was so viele Verbindungen verarbeitet hat.Das Programm läuft auf einem Stock 64 Bit Debian 8 (Jessie) ohne GUI.
Linux version 3.16.0-4-amd64 (debian-kernel@lists.debian.org) (gcc version 4.8.4 (Debian 4.8.4-1) ) #1 SMP Debian 3.16.7-ckt11-1+deb8u6 (2015-11-09)Als Grundgerüst habe ich folgenden Code genommen.
http://www.boost.org/doc/libs/1_58_0/doc/html/boost_asio/example/cpp03/ssl/server.cpp
Das Programm hat zwei Prozesse, wobei ein Prozess nur für das loggen zuständig ist, und eine Log Queue hat.
Im zweiten Prozess läuft dann die Handling der Clients in einem Thread.Das Programm läuft in einer virtuellen Umgebung. Dem System stehen 4 Gigabyte Hauptspeicher zur Verfügung,
davon verwendet das Programm ca. 200 Megabyte, Virtual Memory ca. 400 Megabyte.Wäre es sinnvoller den Teil, welcher die eingehenden Verbindungen akzeptiert, in einem eigenen Prozess laufen zu lassen, und pro ca. 500 offenen Verbindungen einen eigenen Thread zu machen ?
Gruß Till
-
Du hast vergessen uns deinen Quellcode zu zeigen.
-
Hier ein Teil des Quellcodes welcher für das Networking zuständig ist
Sever.hpp/** * @file Server.hpp * * @brief Provides the class to install a tcp server, establishing ssl encrypted connections. */ #ifndef SERVER_H_ #define SERVER_H_ #include <cstdlib> #include <grp.h> #include <iostream> #include <pwd.h> #include <string> #include <boost/asio.hpp> #include <boost/asio/ssl.hpp> #include <boost/bind.hpp> #include "ConsoleAPI.hpp" #include "GatewayConfiguration.hpp" #include "GSSAPI.hpp" #include "Logger.hpp" #include "Session.hpp" #include "Settings.hpp" #include "SQLite.hpp" using namespace std; using namespace ConsoleAPI; /** * @class Server * * @brief Provides all necessary methods to act as a tcp server. * This class is able to run multiple sockets, connecting to one client each. * These sockets are able to provide asynchronous read/write methods to communicate * with the clients in parallel. * All used functions are based on the boost::asio library. */ class Server : virtual private Logger, virtual private Settings, SQLite, public ConsoleAPIIn { ConsoleAPIOut* consoleAPIOut_; boost::asio::io_service ioService_; /**< The service, that handles all sessions. */ boost::asio::ip::tcp::acceptor acceptor_; /**< Acceptor for the new connections. */ boost::asio::ssl::context context_; /**< Configuration for the connections. */ map<unsigned int, Session*> sessionCache_; mutex sessionCacheMutex_; string cipherList_; unsigned int sessionCounter_; /**< Counts the amount of concurrent running sessions. */ boost::asio::deadline_timer renewCacheTimer_; /**< The deadline timer to handle lost connections. */ unsigned int period_; /**< The default time out in seconds. */ struct passwd* pwd_; inline void getCipherList () { cipherList_ = "ECDHE-RSA-AES128-GCM-SHA256"; cipherList_ += ":ECDHE-ECDSA-AES128-GCM-SHA256"; cipherList_ += ":ECDHE-RSA-AES256-GCM-SHA384"; cipherList_ += ":ECDHE-ECDSA-AES256-GCM-SHA384"; cipherList_ += ":DHE-RSA-AES128-GCM-SHA256"; cipherList_ += ":DHE-DSS-AES128-GCM-SHA256"; cipherList_ += ":kEDH+AESGCM"; cipherList_ += ":ECDHE-RSA-AES128-SHA256"; cipherList_ += ":ECDHE-ECDSA-AES128-SHA256"; cipherList_ += ":ECDHE-RSA-AES128-SHA"; cipherList_ += ":ECDHE-ECDSA-AES128-SHA"; cipherList_ += ":ECDHE-RSA-AES256-SHA384"; cipherList_ += ":ECDHE-ECDSA-AES256-SHA384"; cipherList_ += ":ECDHE-RSA-AES256-SHA"; cipherList_ += ":ECDHE-ECDSA-AES256-SHA"; cipherList_ += ":DHE-RSA-AES128-SHA256"; cipherList_ += ":DHE-RSA-AES128-SHA"; cipherList_ += ":DHE-DSS-AES128-SHA256"; cipherList_ += ":DHE-RSA-AES256-SHA256"; cipherList_ += ":DHE-DSS-AES256-SHA"; cipherList_ += ":DHE-RSA-AES256-SHA"; cipherList_ += ":AES128-GCM-SHA256"; cipherList_ += ":AES256-GCM-SHA384"; cipherList_ += ":AES128-SHA256"; cipherList_ += ":AES256-SHA256"; cipherList_ += ":AES128-SHA"; cipherList_ += ":AES256-SHA"; cipherList_ += ":AES"; cipherList_ += ":CAMELLIA"; cipherList_ += ":DES-CBC3-SHA"; cipherList_ += ":!aNULL"; cipherList_ += ":!eNULL"; cipherList_ += ":!EXPORT"; cipherList_ += ":!DES"; cipherList_ += ":!RC4"; cipherList_ += ":!MD5"; cipherList_ += ":!PSK"; cipherList_ += ":!aECDH"; cipherList_ += ":!EDH-DSS-DES-CBC3-SHA"; cipherList_ += ":!EDH-RSA-DES-CBC3-SHA"; cipherList_ += ":!KRB5-DES-CBC3-SHA"; } /** * @brief Yet quite useless, but can be set in the future. * @todo Needs to be checked, if it is necessary at all. * @return The password for the server. */ string get_password(); /** * @brief Enable acceptance of a new session request. */ void startSessionAccept(); /** * @brief Tries to start a new session. * * @param sslSocket The socket for the session to start. * @param error Possible errors, occurring via the construction of the new session. */ void handleAccept(sslSocket_t* sslSocket, const boost::system::error_code& error); /** * @brief The method to use for all sessions, when they kill themselves. */ void sessionDestruction(unsigned int id); bool dropPriviliges(); string resetServer(); string createTestSession(GatewayConfig config); void renewCache(); public: /** * @brief Creates a Server class with the appropriate logger. */ Server(); /** * @brief Destroys the server safely. */ virtual ~Server(); /** * @brief Starts the server. */ void start(); }; #endif /* SERVER_H_ */Server.cpp
#include "Server.hpp" Server::Server() : consoleAPIOut_(NULL), acceptor_( ioService_, boost::asio::ip::tcp::endpoint( boost::asio::ip::tcp::v4(), Settings::getEntry("server.port").empty() ? 443 : stoi(Settings::getEntry("server.port")))), context_(boost::asio::ssl::context::sslv23_server), sessionCounter_(0), renewCacheTimer_(ioService_), period_(stoi(Settings::getEntry("server.autoSyncInterval"))) { acceptor_.set_option(boost::asio::ip::tcp::acceptor::reuse_address(true)); try { if (!dropPriviliges()) { Logger::set_log_entry( trace, std::string(BOOST_CURRENT_FUNCTION) + " : Since the privileges cannot be dropped, the program continues with the current privileges."); } context_.set_options( boost::asio::ssl::context::default_workarounds | boost::asio::ssl::context::no_sslv2 | boost::asio::ssl::context::no_sslv3 | boost::asio::ssl::context::single_dh_use); context_.set_default_verify_paths(); context_.set_password_callback(boost::bind(&Server::get_password, this)); context_.use_private_key_file(GatewayConfiguration::serverPath + Settings::getEntry("server.privateKey"), boost::asio::ssl::context::pem); if (Settings::getEntry("server.intermediateCert").empty()) { context_.use_certificate_file(GatewayConfiguration::serverPath + Settings::getEntry("server.certificate"), boost::asio::ssl::context::pem); } else { ifstream cert(GatewayConfiguration::serverPath + Settings::getEntry("server.certificate")); ifstream inter(GatewayConfiguration::serverPath + Settings::getEntry("server.intermediateCert")); ofstream chain(GatewayConfiguration::serverPath + "certificates/cert_chain.crt", std::ios_base::binary); chain << cert.rdbuf() << inter.rdbuf(); cert.close(); inter.close(); chain.close(); context_.use_certificate_chain_file(GatewayConfiguration::serverPath + "certificates/cert_chain.crt"); } context_.use_tmp_dh_file("dh2048.pem"); cipherList_ = Settings::getEntry("cipherList"); if (cipherList_.empty()) getCipherList(); SSL_CTX_set_cipher_list(context_.native_handle(), cipherList_.c_str()); } catch (std::exception& e) { Logger::set_log_entry( fatal, std::string(BOOST_CURRENT_FUNCTION) + " : Cannot initiate the SSL-configuration!" + " Exception: " + e.what()); exit(0); } } Server::~Server() { } string Server::resetServer() { string ret; acceptor_.cancel(); sessionCacheMutex_.lock(); for (pair<unsigned int, Session*> p : sessionCache_) { p.second->killSession(); } sessionCacheMutex_.unlock(); return ret; } string Server::createTestSession(GatewayConfig config) { string ret; Session* new_session = new Session(ioService_, config, &ret); new_session->killSession(); return ret; } bool Server::dropPriviliges() { pwd_ = getpwnam((const char*) Settings::getEntry("server.dropPrivilegeTo").c_str()); if ((pwd_ != NULL) && !setgid(pwd_->pw_gid) && !setuid(pwd_->pw_uid)) { Logger::set_log_entry(trace, std::string(BOOST_CURRENT_FUNCTION) + " : User privileges have been dropped."); return true; } else { Logger::set_log_entry( error, std::string(BOOST_CURRENT_FUNCTION) + " : User not found: " + Settings::getEntry("server.dropPrivilegeTo")); return false; } } std::string Server::get_password() { return "test"; } void Server::start() { int i = 10; while (i > 0) { try { consoleAPIOut_ = new ConsoleAPIOut(boost::ref(ioService_)); renewCache(); Logger::set_log_entry(trace, std::string(BOOST_CURRENT_FUNCTION) + " : IOService of the server is getting started..."); ioService_.reset(); startSessionAccept(); ioService_.run(); } catch (std::exception& e) { Logger::set_log_entry( fatal, std::string(BOOST_CURRENT_FUNCTION) + " : IOService of the server cannot run!" + " Exception: " + e.what()); } catch (boost::system::error_code& ec) { if (ec == boost::asio::error::basic_errors::broken_pipe) { Logger::set_log_entry( fatal, std::string(BOOST_CURRENT_FUNCTION) + " : IOService of the server crashed because of a socket communication error."); } } i--; } } void Server::startSessionAccept() { sslSocket_t* sslSocket = new sslSocket_t(ioService_, context_); acceptor_.async_accept(sslSocket->lowest_layer(), boost::bind(&Server::handleAccept, this, sslSocket, boost::asio::placeholders::error)); } void Server::handleAccept(sslSocket_t* sslSocket, const boost::system::error_code& err) { if (!err) { sessionCounter_++; Session* new_session = new Session(sslSocket, boost::bind(&Server::ConsoleAPIIn::processCommand, this, _1), boost::bind(&Server::sessionDestruction, this, _1), sessionCounter_); sessionCacheMutex_.lock(); sessionCache_.insert(make_pair(sessionCounter_, new_session)); sessionCacheMutex_.unlock(); new_session->start(); Logger::set_log_entry( trace, std::string(BOOST_CURRENT_FUNCTION) + " : New session has been started. (" + sslSocket->lowest_layer().remote_endpoint().address().to_string() + ")"); } else if (err != boost::asio::error::operation_aborted) { delete sslSocket; Logger::set_log_entry(trace, std::string(BOOST_CURRENT_FUNCTION) + " : Acception of new connections has been stopped."); return; } else { delete sslSocket; Logger::set_log_entry( error, std::string(BOOST_CURRENT_FUNCTION) + " : Cannot establish a tcp connection. " + err.message()); } this->startSessionAccept(); } void Server::renewCache() { Logger::set_log_entry(trace, std::string(BOOST_CURRENT_FUNCTION) + " : Updating cache."); if (consoleAPIOut_) { consoleAPIOut_->getGatewayConfigs(); consoleAPIOut_->getClientList(false); } renewCacheTimer_.expires_from_now(boost::posix_time::minutes(period_)); renewCacheTimer_.async_wait(boost::bind(&Server::renewCache, this)); } void Server::sessionDestruction(unsigned int id) { sessionCacheMutex_.lock(); map<unsigned int, Session*>::iterator res = sessionCache_.find(id); if (res != sessionCache_.end()) { sessionCache_.erase(res); } else { Logger::set_log_entry(error, std::string(BOOST_CURRENT_FUNCTION) + " : Session without a valid id is getting deleted."); } sessionCacheMutex_.unlock(); }Session.hpp
#ifndef SESSION_HPP_ #define SESSION_HPP_ #include <chrono> #include <future> #include <string> #include <cstddef> #include <cstdlib> #include <iostream> #include <memory> #include <map> #include <mutex> #include <thread> #include <boost/asio.hpp> #include <boost/asio/deadline_timer.hpp> #include <boost/asio/ssl.hpp> #include <boost/bind.hpp> #include <boost/function.hpp> #include <boost/algorithm/string.hpp> #include "Base64.hpp" #include "ConsoleAPI.hpp" #include "GatewayConfiguration.hpp" #include "Crypto.hpp" #include "GSSAPI.hpp" #include "Logger.hpp" #include "Settings.hpp" #include "Session.hpp" #include "SQLite.hpp" typedef boost::asio::ssl::stream<boost::asio::ip::tcp::socket> sslSocket_t; /**< Provides some readability. As stated in the name, it provides a ssl socket. */ using namespace std; using namespace GatewayConfiguration; using namespace ConsoleAPI; /** * @class Session * * @brief Provides all necessary methods to handle a connection to a client via tcp. * * * @version 1.0 * * This class handles all IO-functions to a tcp client. * It also takes care of time outs and all occurring errors in the connection. */ class Session : virtual private Logger, virtual private Settings, GSSAPI, Crypto, ConsoleAPIOut { unsigned int id_; boost::asio::ssl::context context_; /**< Configuration for Remote connections. */ sslSocket_t* sslSocket_; /**< The socket for the ssl connection. */ sslSocket_t* sslSocketExchange_; static const chrono::seconds connectTimeout_; /**< Duration to wait for an established connection. */ future<void> connectionTimer_; char* data_; /**< A buffer for all incoming data on the stream. */ string readBuffer_; /**< A buffer to store the messages. */ string writeBuffer_; /**< A buffer for the message to write on the stream socket. */ const size_t defaultBufferSize_; /**< Default size for the buffer. */ string request_; /**< A constructed request for the clients. */ boost::asio::deadline_timer dlTimer_; /**< The deadline timer to handle lost connections. */ static const unsigned int timeOut_; /**< The default time out in seconds. */ boost::function<void(unsigned int)> serverSessionDestruction_; /**< The method to call on the server, when a session is destructed. */ unsigned char protocolVersion_; string command_; string deviceId_; string deviceType_; string user_; string pwd_; void connectAsync();bool connectToExchange(); DeviceConfig deviceConfig_; bool isConsoleCommand_; boost::function<string(string JSONCommand)> serverConsoleAPIProcessCommand_; /** * @brief Checks, if all asynchronous waits are done. */ void expireDeadline(); /** * @brief Checks, if the request message is complete. * * @param bytes_transferred The amount of bytes transfered with the last read. * @return The appropriate client id. -1, if the request is not complete yet. */ bool isCompleteRequest(size_t bytes_transferred, sslSocket_t* sslSocket); /** * @brief Handles The handshake with a client. * @param error An arbitrary error while handshaking. */ void handleHandshake(const boost::system::error_code& error); /** * @brief Handles the reading from the connection stream. * @param error An arbitrary error while reading. * @param bytes_transferred Amount of bytes transfered via stream. */ void handleRead(const boost::system::error_code& error, size_t bytes_transferred, sslSocket_t* sslSocket); /** * @brief Handles the writing to the connection stream. * @param error An arbitrary error while handshaking. */ void handleWrite(const boost::system::error_code& error, sslSocket_t* sslSocket); /** * @brief Handle the timeout. * @param err The error from the previous call. */ void handleTimeout(const boost::system::error_code& err); /** * @brief Handles the closure of a session. * @param err The error from the previous call. */ void handleShutdown(const boost::system::error_code& err, sslSocket_t* sslSocket); /** * @brief Read asynchronous from the socket stream. */ void readSocketStreamToBuffer(sslSocket_t* sslSocket); /** * @brief Writes the message into the SSLHandler::writeBuffer_. * @param message Message, that shalsize_t defaultBufferSize = 1024size_t defaultBufferSize = 1024 */ void writeToSocketStream(string message, sslSocket_t* sslSocket); public: /** * @brief Creates a Session class. * @param sslSocket The socket to communicate on. * @param serverSessionDestructionPtr A pointer to the method that shall be called on the server class, when a session is getting deleted. */ Session(sslSocket_t* sslSocket, boost::function<string (string)> serverConsoleAPIProcessCommand, boost::function<void(unsigned int)> serverSessionDestructionPtr, unsigned int id, size_t defaultBufferSize = 1024); Session(boost::asio::io_service& ioService, GatewayConfig gC, string* message); /** * @brief Kills a session safely. */ ~Session(); /** * @brief Starts this session. */ void start(); /** * @brief Kill the session in a controlled way. */ void killSession(); }; #endif /* SESSION_HPP_ */Session.cpp
#include "Session.hpp" const unsigned int Session::timeOut_ = 3600; const chrono::seconds Session::connectTimeout_(5); Session::Session(sslSocket_t* sslSocket, boost::function<string(string)> serverConsoleAPIProcessCommand, boost::function<void(unsigned int)> serverSessionDestructionPtr, unsigned int id, size_t defaultBufferSize) : ConsoleAPIOut(sslSocket->get_io_service()), id_(id), context_(boost::asio::ssl::context::sslv23_client), sslSocket_(sslSocket), sslSocketExchange_(new sslSocket_t(sslSocket->get_io_service(), context_)), data_(new char[defaultBufferSize]), defaultBufferSize_(defaultBufferSize), dlTimer_(sslSocket_->get_io_service()), serverSessionDestruction_(serverSessionDestructionPtr), protocolVersion_(0), isConsoleCommand_(false), serverConsoleAPIProcessCommand_(serverConsoleAPIProcessCommand) { } Session::Session(boost::asio::io_service &ioService, GatewayConfig gC, string* message) : ConsoleAPIOut(ioService), id_(0), context_(boost::asio::ssl::context::sslv23_client), sslSocket_(NULL), sslSocketExchange_(new sslSocket_t(ioService, context_)), data_(new char[0]), defaultBufferSize_(0), dlTimer_(ioService), serverSessionDestruction_(NULL), protocolVersion_(0), isConsoleCommand_(false), serverConsoleAPIProcessCommand_(NULL) { Logger::set_log_entry(trace, std::string(BOOST_CURRENT_FUNCTION) + " : Test session has been created."); GSSAPI::gatewayConfig_ = gC; connectionTimer_ = async(launch::async, &Session::connectAsync, this); if (!connectToExchange()) { *message = "Test session is unable to connect to exchange. "; } } Session::~Session() { if (sslSocketExchange_) sslSocketExchange_->lowest_layer().close(); if (sslSocket_) sslSocket_->lowest_layer().close(); delete sslSocket_; delete sslSocketExchange_; delete data_; if (serverSessionDestruction_) serverSessionDestruction_(id_); dlTimer_.cancel(); Logger::set_log_entry(trace, std::string(BOOST_CURRENT_FUNCTION) + " : Session has been closed properly."); } void Session::start() { try { sslSocket_->async_handshake(boost::asio::ssl::stream_base::server, boost::bind(&Session::handleHandshake, this, boost::asio::placeholders::error)); dlTimer_.expires_from_now(boost::posix_time::seconds(Session::timeOut_)); dlTimer_.async_wait(boost::bind(&Session::handleTimeout, this, boost::asio::placeholders::error)); } catch (std::exception& e) { Logger::set_log_entry( critical, std::string(BOOST_CURRENT_FUNCTION) + " : Session cannot get started." + " Exception: " + e.what()); killSession(); } } void Session::connectAsync() { try { boost::asio::ip::tcp::resolver resolver(sslSocketExchange_->get_io_service()); boost::asio::ip::tcp::resolver::query query(GSSAPI::gatewayConfig_.remoteHost1, "443"); boost::asio::connect(sslSocketExchange_->lowest_layer(), resolver.resolve(query)); sslSocketExchange_->handshake(boost::asio::ssl::stream_base::client); } catch (std::exception& e) { Logger::set_log_entry( critical, std::string(BOOST_CURRENT_FUNCTION) + " : Connection to the exchange server can not be established." + " Exception: " + e.what()); killSession(); } } bool Session::connectToExchange() { try { if (connectionTimer_.wait_for(Session::connectTimeout_) == future_status::ready) { return true; } else { Logger::set_log_entry( warning, std::string(BOOST_CURRENT_FUNCTION) + " : Connection attempt to " + GSSAPI::gatewayConfig_.remoteHost1 + " timed out."); } } catch (exception& e) { Logger::set_log_entry( error, std::string(BOOST_CURRENT_FUNCTION) + " : Unable to connect to " + GSSAPI::gatewayConfig_.remoteHost1 + " Exception: " + e.what()); } return false; } void Session::handleHandshake(const boost::system::error_code& err) { if (!err) { try { Session::readSocketStreamToBuffer(sslSocket_); expireDeadline(); return; } catch (std::exception& e) { Logger::set_log_entry( critical, std::string(BOOST_CURRENT_FUNCTION) + " : Something went wrong during the SSL handshake." + " Exception: " + e.what()); killSession(); } } else if (err.category() == boost::asio::error::get_ssl_category()) { Logger::set_log_entry( warning, std::string(BOOST_CURRENT_FUNCTION) + " : An OpenSSL error: " + err.category().name() + ":" + err.message()); } else { Logger::set_log_entry( error, std::string(BOOST_CURRENT_FUNCTION) + " : An error occurred during the handshake: " + err.message()); } killSession(); } void Session::handleRead(const boost::system::error_code& err, size_t bytes_transferred, sslSocket_t* sslSocket) { if (!err) { expireDeadline(); if (isCompleteRequest(bytes_transferred, sslSocket)) { try { if (isConsoleCommand_) { size_t bodySize = readBuffer_.find("\n"); if (bodySize != readBuffer_.npos) { Logger::set_log_entry(trace, std::string(BOOST_CURRENT_FUNCTION) + " : Console command has been received."); //do stuff with that command killSession(); } } else if (sslSocket_ == sslSocket) { Logger::set_log_entry( trace, std::string(BOOST_CURRENT_FUNCTION) + " : Request from DeviceId " + deviceId_ + " complete."); size_t pos = readBuffer_.find("Authorization: Basic"); if (pos != readBuffer_.npos) { Logger::set_log_entry( trace, std::string(BOOST_CURRENT_FUNCTION) + " : Request from DeviceId " + deviceId_ + " uses 2-factor authentication."); request_ = Base64::decode(string(&readBuffer_[pos + 21], readBuffer_.find("\n", pos) - pos - 22)); size_t sep = request_.find(":"); user_ = string(&request_[0], sep); pwd_ = string(&request_[sep + 1], request_.length() - sep - 1); Logger::set_log_entry( trace, std::string(BOOST_CURRENT_FUNCTION) + " : User " + user_ + " requests the gateway services."); connectionTimer_ = async(launch::async, &Session::connectAsync, this); if (connectToExchange()) { writeToSocketStream(readBuffer_, sslSocketExchange_); } else { killSession(); } } } else { Logger::set_log_entry(trace, std::string(BOOST_CURRENT_FUNCTION) + " : Request from exchange complete."); writeToSocketStream(readBuffer_, sslSocket_); } } catch (std::exception& e) { Logger::set_log_entry( critical, std::string(BOOST_CURRENT_FUNCTION) + " : Something went wrong, while processing the read request." + " Exception: " + e.what()); killSession(); } } else { Session::readSocketStreamToBuffer(sslSocket); } } else if ((err == boost::asio::error::eof) || (err == boost::asio::error::connection_reset)) { Logger::set_log_entry(trace, std::string(BOOST_CURRENT_FUNCTION) + " : Session has been closed on client side."); killSession(); } else if (err == boost::asio::error::operation_aborted) { Logger::set_log_entry( trace, std::string(BOOST_CURRENT_FUNCTION) + " : Reading has been aborted (Probably to shutdown the stream)."); } else if ((err.category() == boost::asio::error::get_ssl_category()) && (err.value() == ERR_PACK(ERR_LIB_SSL, 0, SSL_R_SHORT_READ))) { Logger::set_log_entry( warning, std::string(BOOST_CURRENT_FUNCTION) + " : Short Read. Probably a disconnect from the client."); killSession(); } else { Logger::set_log_entry( error, std::string(BOOST_CURRENT_FUNCTION) + " : An error occurred while reading the stream: " + err.message()); killSession(); } } void Session::handleWrite(const boost::system::error_code& err, sslSocket_t* sslSocket) { if (!err) { readBuffer_ = ""; readSocketStreamToBuffer(sslSocket); } else if (err == boost::asio::error::operation_aborted) { Logger::set_log_entry( trace, std::string(BOOST_CURRENT_FUNCTION) + " : Writing has been aborted (Probably to shutdown the stream)."); } else if (err == boost::asio::error::connection_reset) { Logger::set_log_entry( trace, std::string(BOOST_CURRENT_FUNCTION) + " : Connection has been reset (Probably closing an old socket)."); } else { Logger::set_log_entry( error, std::string(BOOST_CURRENT_FUNCTION) + " : An error occurred while writing the stream: " + err.message()); killSession(); } } void Session::handleTimeout(const boost::system::error_code& err) { if (err != boost::asio::error::operation_aborted) { Logger::set_log_entry(trace, std::string(BOOST_CURRENT_FUNCTION) + " : Session timed out."); killSession(); } } void Session::handleShutdown(const boost::system::error_code& err, sslSocket_t* sslSocket) { if (!err) { Logger::set_log_entry(trace, std::string(BOOST_CURRENT_FUNCTION) + " : Session has been shutdown properly."); } else if ((err == boost::asio::error::broken_pipe) || (err == boost::asio::error::connection_reset)) { Logger::set_log_entry( trace, std::string(BOOST_CURRENT_FUNCTION) + " : Client has roughly shutdown the connection. Session has been shutdown."); } else if ((err == boost::asio::error::eof)) { Logger::set_log_entry(trace, std::string(BOOST_CURRENT_FUNCTION) + " : End of file. Session has been shutdown."); } else if ((err.category() == boost::asio::error::get_ssl_category()) && (err.value() == ERR_PACK(ERR_LIB_SSL, 0, SSL_R_SHORT_READ))) { Logger::set_log_entry( warning, std::string(BOOST_CURRENT_FUNCTION) + " : Short Read. Probably a disconnect from the client. Session has been shutdown."); } else { Logger::set_log_entry( error, std::string(BOOST_CURRENT_FUNCTION) + " : An error occurred during the shutdown: " + err.message() + ". Session has been shutdown."); } delete this; } void Session::killSession() { try { if (sslSocketExchange_ && sslSocketExchange_->lowest_layer().is_open()) sslSocketExchange_->lowest_layer().cancel(); if (sslSocket_ && sslSocket_->lowest_layer().is_open()) { sslSocket_->lowest_layer().cancel(); sslSocket_->async_shutdown( boost::bind(&Session::handleShutdown, this, boost::asio::placeholders::error, sslSocket_)); } } catch (std::exception& e) { Logger::set_log_entry( critical, std::string(BOOST_CURRENT_FUNCTION) + " : Something went wrong, while the Session should be killed." + " Exception: " + e.what()); delete this; } } void Session::expireDeadline() { if (dlTimer_.expires_from_now(boost::posix_time::seconds(Session::timeOut_)) > 0) { dlTimer_.async_wait(boost::bind(&Session::handleTimeout, this, boost::asio::placeholders::error)); } } bool Session::isCompleteRequest(size_t bytes_transferred, sslSocket_t* sslSocket) { try { readBuffer_.append(&data_[0], bytes_transferred); size_t bodyStart = readBuffer_.find("\r\n\r\n"); if (bodyStart != readBuffer_.npos) { size_t bodySize = readBuffer_.rfind("Content-Length: ", bodyStart); bodySize = (bodySize == readBuffer_.npos ? 0 : atoi(&readBuffer_[bodySize + 16])); if (bodyStart + 4 + bodySize - readBuffer_.length() == 0) { if (sslSocket == sslSocketExchange_) return true; if (!deviceId_.empty()) return true; bodySize = readBuffer_.find("\n"); if (bodySize != readBuffer_.npos) { request_ = string(&readBuffer_[0], bodySize); bodyStart = request_.find("?"); if (bodyStart != request_.npos) { bodySize = request_.find("&", bodyStart + 1); if (bodySize != request_.npos) { Logger::set_log_entry(trace, std::string(BOOST_CURRENT_FUNCTION) + " : Processing plain query: " + request_); while (bodySize != (request_.length() - 9)) { bodyStart++; bodySize = request_.find("&", bodyStart); if (bodySize == request_.npos) bodySize = request_.length() - 9; if (request_.find("User=", bodyStart) == bodyStart) { user_ = string(&request_[bodyStart + 5], bodySize - bodyStart - 5); } else if (request_.find("Cmd=", bodyStart) == bodyStart) { command_ = string(&request_[bodyStart + 4], bodySize - bodyStart - 4); } else if (request_.find("DeviceId=", bodyStart) == bodyStart) { deviceId_ = string(&request_[bodyStart + 9], bodySize - bodyStart - 9); boost::to_upper(deviceId_); } else if (request_.find("DeviceType=", bodyStart) == bodyStart) { deviceType_ = string(&request_[bodyStart + 11], bodySize - bodyStart - 11); } bodyStart = bodySize; } } else { Logger::set_log_entry(trace, std::string(BOOST_CURRENT_FUNCTION) + " : Processing base64 query: " + request_); request_ = Base64::decode(string(&request_[bodyStart + 1], request_.length() - bodyStart - 11)); protocolVersion_ = request_[0]; command_ = commandCodeToString(request_[1]); //just a mapping function unsigned char idLength = request_[4]; deviceId_ = string(&request_[5], idLength); ostringstream s; s << std::hex << setfill('0'); for (size_t i = 0; i < deviceId_.length(); i++) { s << std::setw(2) << ((unsigned int) (unsigned char) deviceId_[i]); } deviceId_ = s.str(); boost::to_upper(deviceId_); unsigned char policyLength = request_[idLength + 5]; deviceType_ = string(&request_[idLength + 7 + policyLength], (unsigned char) request_[idLength + 6 + policyLength]); } Logger::set_log_entry( trace, std::string(BOOST_CURRENT_FUNCTION) + " : Query for DeviceId " + deviceId_ + " has been processed."); } } return true; } } else if (readBuffer_.find("Console") == 0) { isConsoleCommand_ = true; size_t bodySize = readBuffer_.find("\n"); if (bodySize != readBuffer_.npos) { return true; } } } catch (std::exception& e) { Logger::set_log_entry( critical, std::string(BOOST_CURRENT_FUNCTION) + " : Something went wrong, while processing the request. Exception: " + e.what()); killSession(); } return false; } void Session::readSocketStreamToBuffer(sslSocket_t* sslSocket) { if (sslSocket && sslSocket->lowest_layer().is_open()) sslSocket->async_read_some( boost::asio::buffer(data_, defaultBufferSize_), boost::bind(&Session::handleRead, this, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred, sslSocket)); } void Session::writeToSocketStream(string message, sslSocket_t* sslSocket) { writeBuffer_ = message; if (sslSocket && sslSocket->lowest_layer().is_open()) boost::asio::async_write(*sslSocket, boost::asio::buffer(writeBuffer_), boost::bind(&Session::handleWrite, this, boost::asio::placeholders::error, sslSocket)); } }Den Teil in der Session welcher GSSAPI::gatewayConfig_ setze habe ich aus Übersichtlichkeitsgründen entfernt, die Variable ist aber stets richtig gesetzt,
und der Aufruf der Funktion lädt den Wert für gatewayConfig aus dem Speicher.
-
Auch der locale Verbindungsaufbau mittels openssl s_client -connect localhost:443 dauert über 10 Sekunden.
Einfach mal an vielen Stellen Logging reinknallen und eingrenzen was überhaupt so lange dauert.
10 Sekunden sind ja extrem viel, da sollte man über die Timestamps im Logfile schon gut sehen können wo die Zeit draufgeht.
