Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package ndpi for openSUSE:Factory checked in at 2021-08-26 23:14:57 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/ndpi (Old) and /work/SRC/openSUSE:Factory/.ndpi.new.1899 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "ndpi" Thu Aug 26 23:14:57 2021 rev:10 rq:914423 version:4.0 Changes: -------- --- /work/SRC/openSUSE:Factory/ndpi/ndpi.changes 2021-04-24 23:10:38.931521712 +0200 +++ /work/SRC/openSUSE:Factory/.ndpi.new.1899/ndpi.changes 2021-08-26 23:15:57.324216788 +0200 @@ -1,0 +2,148 @@ +Thu Aug 26 16:30:53 UTC 2021 - Dirk Stoecker <[email protected]> + +- Add conflicts for ndpi-common package, as version 3 did not follow + packaging guidelines fully + +------------------------------------------------------------------- +Thu Aug 26 09:15:54 UTC 2021 - Martin Hauke <[email protected]> + +- Create -common subpackage + +------------------------------------------------------------------- +Sun Aug 22 12:48:59 UTC 2021 - Martin Hauke <[email protected]> + +- Update to version 4.0 + New Features + * Add API for computing RSI (Relative Strenght Index) + * Add GeoIP support + * Add fragments management + * Add API for jitter calculation + * Add single exponential smoothing API + * Add timeseries forecasting support implementing Holt-Winters + with confidence interval + * Add support for MAC to radi tree and expose the full API to + applications + * Add JA3+, with ALPN and elliptic curve + * Add double exponential smoothing implementation + * Extended API for managing flow risks + * Add flow risk score + * New flow risks: + + Desktop or File Sharing Session + + HTTP suspicious content (useful for tracking trickbot) + + Malicious JA3 + + Malicious SHA1 + + Risky domain + + Risky AS + + TLS Certificate Validity Too Long + + TLS Suspicious Extension + New Supported Protocols and Services + * New protocols: + + AmongUs + + AVAST SecureDNS + + CPHA (CheckPoint High Availability Protocol) + + DisneyPlus + + DTLS + + Genshin Impact + + HP Virtual Machine Group Management (hpvirtgrp) + + Mongodb + + Pinterest + + Reddit + + Snapchat VoIP calls + + Tumblr + + Virtual Asssitant (Alexa, Siri) + + Z39.50 + * Add protocols to HTTP as subprotocols + * Add detection of TLS browser type + * Add connectionless DCE/RPC detection + Improvements + * 2.5x speed bump. Example ndpiReader with a long mixed pcap + v3.4 - nDPI throughput: 1.29 M pps / 3.35 Gb/sec + v4.0 - nDPI throughput: 3.35 M pps / 8.68 Gb/sec + * Improve detection/dissection of: + + AnyDesk + + DNS + + Hulu + + DCE/RPC (avoid false positives) + + dnscrypt + + Facebook (add new networks) + + Fortigate + + FTP Control + + HTTP + - Fix user-agent parsing + - Fix logs when NDPI_ENABLE_DEBUG_MESSAGES is defined + + IEC104 + + IEC60870 + + IRC + + Netbios + + Netflix + + Ookla speedtest (detection over IPv6) + + openspeedtest.com + + Outlook / MicrosoftMail + + QUIC + - update to draft-33 + - improve handling of SNI + - support for fragmented Client Hello + - support for DNS-over-QUIC + + RTSP + + RTSP via HTTP + + SNMP (reimplemented) + + Skype + + SSH + + Steam (Steam Datagram Relay - SDR) + + STUN (avoid false positives, improved Skype detection) + + TeamViewer (add new hosts) + + TOR (update hosts) + + TLS + - Certificate Subject matching + - Check for common ALPNs + - Reworked fingerprint calculation + - Fix extraction for TLS signature algorithms + - Fix ClientHello parsing + + UPnP + + wireguard + + Improve DGA detection + + Improve JA3 + + Improve Mining detection + + Improve string matching algorithm + + Improve ndpi_pref_enable_tls_block_dissection + + Optimize speed and memory size + + Update ahocorasick library + + Improve subprotocols detection + Fixes + * Fix partial application matching + * Fix multiple segfault and leaks + * Fix uninitialized memory use + * Fix release of patterns allocated in ndpi_add_string_to_automa + * Fix return value of ndpi_match_string_subprotocol + * Fix setting of flow risks on 32 bit machines + * Fix TLS certificate threshold + * Fix a memory error in TLS JA3 code + * Fix false positives in Z39.50 + * Fix off-by-one memory error for TLS-JA3 + * Fix bug in ndpi_lru_find_cache + * Fix invalid xbox and playstation port guesses + * Fix CAPWAP tunnel decoding + * Fix parsing of DLT_PPP datalink type + * Fix dissection of QUIC initial packets coalesced with 0-RTT one + * Fix parsing of GTP headers + * Add bitmap boundary checks + Misc + * Update download category name + * Update category labels + * Renamed Skype in Skype_Teams (the protocol is now shared across + these apps) + * Add IEC analysis wireshark plugin + * Flow risk visualization in Wireshark + * ndpiReader + + add statistics about nDPI performance + + fix memory leak + + fix collecting of risks statistics + * Move installed libraries from /usr/local to /usr + * Improve NDPI_API_VERSION generation + * Update ndpi_ptree_match_addr prototype +- Add patches (for compatibility with ntopng 5.0): + * 0001-Added-ability-to-report-whether-a-protocol-is-encryp.patch + * 0002-Report-whether-a-protocol-is-encrypted.patch + * 0003-Firs-crash-on-ARM-during-steam-protocol-dissection.patch + +------------------------------------------------------------------- Old: ---- ndpi-3.4.tar.gz New: ---- 0001-Added-ability-to-report-whether-a-protocol-is-encryp.patch 0002-Report-whether-a-protocol-is-encrypted.patch 0003-Firs-crash-on-ARM-during-steam-protocol-dissection.patch ndpi-4.0.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ ndpi.spec ++++++ --- /var/tmp/diff_new_pack.N5m4LS/_old 2021-08-26 23:15:57.980216233 +0200 +++ /var/tmp/diff_new_pack.N5m4LS/_new 2021-08-26 23:15:57.984216230 +0200 @@ -2,7 +2,7 @@ # spec file for package ndpi # # Copyright (c) 2021 SUSE LLC -# Copyright (c) 2017, Martin Hauke <[email protected]> +# Copyright (c) 2017-2021, Martin Hauke <[email protected]> # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -21,9 +21,9 @@ %bcond_without hyperscan %endif -%define sover 3 +%define sover 4 Name: ndpi -Version: 3.4 +Version: 4.0 Release: 0 Summary: Extensible deep packet inspection library # wireshark/ndpi.lua is GPL-3.0-or-later @@ -31,6 +31,12 @@ Group: Development/Libraries/C and C++ URL: https://github.com/ntop/nDPI Source: https://github.com/ntop/nDPI/archive/%{version}.tar.gz#/%{name}-%{version}.tar.gz +# PATCH-FIX-UPSTREAM 0001-Added-ability-to-report-whether-a-protocol-is-encryp.patch # ntopng 5.0 needs this from the ndpi 4.0-stable branch +Patch0: 0001-Added-ability-to-report-whether-a-protocol-is-encryp.patch +# PATCH-FIX-UPSTREAM 0002-Report-whether-a-protocol-is-encrypted.patch # ntopng 5.0 needs this from the ndpi 4.0-stable branch +Patch1: 0002-Report-whether-a-protocol-is-encrypted.patch +# PATCH-FIX-UPSTREAM 0003-Firs-crash-on-ARM-during-steam-protocol-dissection.patch +Patch2: 0003-Firs-crash-on-ARM-during-steam-protocol-dissection.patch BuildRequires: autoconf BuildRequires: automake BuildRequires: gcc-c++ @@ -51,6 +57,7 @@ %package -n libndpi%{sover} Summary: Extensible deep packet inspection library Group: System/Libraries +Requires: ndpi-common %description -n libndpi%{sover} nDPI is a ntop-maintained superset of the OpenDPI library. It extends @@ -86,8 +93,24 @@ This package contains the ndpiReader binary. +%package -n ndpi-common +Summary: Common files used by nDPI +Group: Development/Libraries/C and C++ +# version 3 rpm did not yet follow rules correctly +Conflicts: libndpi3 + +%description -n ndpi-common +nDPI is a ntop-maintained superset of the OpenDPI library. It extends +the original library by adding new protocols that are otherwise +available only on the paid version of OpenDPI. + +This package contains common files used by nDPI. + %prep %setup -q -n nDPI-%{version} +%patch0 -p1 +%patch1 -p1 +%patch2 -p1 %build sh autogen.sh @@ -101,16 +124,12 @@ %install %make_install PREFIX=%{_prefix} prefix=%{_prefix} libdir=%{_libdir} rm -f %{buildroot}/%{_libdir}/libndpi.a -rm -rf %{buildroot}/%{_sbindir}/ndpi +rm -f %{buildroot}/%{_sbindir}/ndpi %post -n libndpi%{sover} -p /sbin/ldconfig %postun -n libndpi%{sover} -p /sbin/ldconfig %files -n libndpi%{sover} -%license COPYING -%doc CHANGELOG.md README.md README.nDPI README.protocols -%doc doc/nDPI_QuickStartGuide.pdf -%{_datadir}/%{name} %{_libdir}/libndpi.so.%{sover}* %files -n libndpi-devel @@ -122,4 +141,10 @@ %{_bindir}/ndpiReader %doc wireshark +%files -n ndpi-common +%license COPYING +%doc CHANGELOG.md README.md README.nDPI README.protocols +%doc doc/nDPI_QuickStartGuide.pdf +%{_datadir}/%{name} + %changelog ++++++ 0001-Added-ability-to-report-whether-a-protocol-is-encryp.patch ++++++ ++++ 1343 lines (skipped) ++++++ 0002-Report-whether-a-protocol-is-encrypted.patch ++++++ >From 5f286a17c1ecb88b06ee069650fa73f7565165dc Mon Sep 17 00:00:00 2001 From: Luca Deri <[email protected]> Date: Sat, 7 Aug 2021 17:35:34 +0200 Subject: [PATCH 2/3] Report whether a protocol is encrypted --- example/ndpiReader.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/example/ndpiReader.c b/example/ndpiReader.c index b4434650..053dc2ec 100644 --- a/example/ndpiReader.c +++ b/example/ndpiReader.c @@ -1368,6 +1368,9 @@ static void printFlow(u_int32_t id, struct ndpi_flow_info *flow, u_int16_t threa ndpi_protocol2name(ndpi_thread_info[thread_id].workflow->ndpi_struct, flow->detected_protocol, buf1, sizeof(buf1))); + fprintf(out, "[%s]", + ndpi_is_encrypted_proto(ndpi_thread_info[thread_id].workflow->ndpi_struct, flow->detected_protocol) ? "Encrypted" : "ClearText"); + if(flow->detected_protocol.category != 0) fprintf(out, "[cat: %s/%u]", ndpi_category_get_name(ndpi_thread_info[thread_id].workflow->ndpi_struct, -- 2.32.0 ++++++ 0003-Firs-crash-on-ARM-during-steam-protocol-dissection.patch ++++++ >From 817c00b65ab4d0bf78927e494227ac5308417f91 Mon Sep 17 00:00:00 2001 From: Luca Deri <[email protected]> Date: Wed, 18 Aug 2021 11:33:33 +0200 Subject: [PATCH 3/3] Firs crash on ARM during steam protocol dissection --- src/lib/protocols/steam.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/src/lib/protocols/steam.c b/src/lib/protocols/steam.c index 8cd3ec41..53bbfc6a 100644 --- a/src/lib/protocols/steam.c +++ b/src/lib/protocols/steam.c @@ -114,14 +114,19 @@ static void ndpi_check_steam_udp1(struct ndpi_detection_module_struct *ndpi_stru } /* Check for Steam Datagram Relay (SDR) packets. */ - if (payload_len > 8 && - ndpi_ntohll(get_u_int64_t(packet->payload, 0)) == 0x0101736470696e67 /* "\x01\x01sdping" */) - { - NDPI_LOG_INFO(ndpi_struct, "found STEAM (Steam Datagram Relay)\n"); - ndpi_int_steam_add_connection(ndpi_struct, flow); - return; - } + if (payload_len > 8) { + u_int64_t n; + /* Necessary as simple cast crashes on ARM */ + memcpy(&n, packet->payload, sizeof(u_int64_t)); + + if(ndpi_ntohll(n) == 0x0101736470696e67 /* "\x01\x01sdping" */) { + NDPI_LOG_INFO(ndpi_struct, "found STEAM (Steam Datagram Relay)\n"); + ndpi_int_steam_add_connection(ndpi_struct, flow); + return; + } + } + /* Check if we so far detected the protocol in the request or not. */ if (flow->steam_stage1 == 0) { NDPI_LOG_DBG2(ndpi_struct, "STEAM stage 0: \n"); -- 2.32.0 ++++++ ndpi-3.4.tar.gz -> ndpi-4.0.tar.gz ++++++ /work/SRC/openSUSE:Factory/ndpi/ndpi-3.4.tar.gz /work/SRC/openSUSE:Factory/.ndpi.new.1899/ndpi-4.0.tar.gz differ: char 13, line 1
