---------------------------------------------------------------------------- S H O R E W A L L 5 . 2 . 0 . 4 ------------------------------- M A Y 2 0 , 2 0 1 8 ---------------------------------------------------------------------------- I. PROBLEMS CORRECTED IN THIS RELEASE II. KNOWN PROBLEMS REMAINING III. NEW FEATURES IN THIS RELEASE IV. MIGRATION ISSUES V. PROBLEMS CORRECTED AND NEW FEATURES IN PRIOR RELEASES ---------------------------------------------------------------------------- I. P R O B L E M S C O R R E C T E D I N T H I S R E L E A S E ---------------------------------------------------------------------------- 5.2.0.4 1) The 'lost carrier' change in 5.0.2.3 did not play well with link monitors like FooLSM. When carrier was restored, the link monitor could be unable to detect that the interface was working again. This has been corrected so that the monitor can detect the availability of the link. 2) If - DYNAMIC_BLACKLIST=ipset...,src-dst... with logging specified - dbl=src_dst appears in the OPTIONS column of an interface then compilation previously produced a series of Perl runtime diagnostics Use of uninitialized value $to in split at /usr/share/shorewall/Shorewall/Chains.pm line 2769. Use of uninitialized value $target in hash element at /usr/share/Shorewall/Chains.pm line 2770. Use of uninitialized value $target in hash element at /usr/share/shorewall/Shorewall/Chains.pm line 2771. Use of uninitialized value $to in concatenation (.) or string at /usr/share/shorewall/Shorewall/Chains.pm line 2771. and possibly the message ERROR: Unknown rule target () ... That problem has been corrected. 5.2.0.3 1) The 'update' command previously did not replace 'Drop' or 'Reject' in the setting of BLACKLIST_DEFAULT. That has been corrected. 2) The 'update' command (and automatic conversion of the masq file) previously failed to handle variables of the form ${...} correctly, resulting in "Invalid column/value pair" errors. That has been corrected. Note, however, that the converted file will have the braces ("{" and "}") removed. 3) If AUTOMAKE was not specified in shorewall[6].conf, the following Perl diagnostic was issued: Use of uninitialized value $val in pattern match (m//) at /usr/share/shorewall/Shorewall/Config.pm line 6602 That has been corrected. 4) Previously, if an ethernet provider interface lost carrier, an attempt to disable the interface could result in an error similar to this: Error: "nexthop" or end of line is expected instead of "linkdown" ERROR: Command "ip -4 route replace table 250 default nexthop via 192.168.0.1 dev eth2 weight 1 linkdown" Failed That has been corrected. 5.2.0.2 1) The 'show saves' command previously failed when there were no saved configurations. That has been corrected. 2) The 'safe-' commands previously failed with the error: /usr/sbin/shorewall: 1194: /usr/sbin/shorewall: read_yesno_with_timeout: not found That has been corrected. 3) When the -c option was specified with the 'compile' command, and 'AUTOMAKE=No' or 'AUTOMAKE=', the command previously failed with errors such as: usr/sbin/shorewall: 415: [: =: unexpected operator /usr/bin/find: Expected a positive decimal integer argument to -maxdepth, but got ‘-type’ /usr/sbin/shorewall: 415: [: =: unexpected operator /usr/bin/find: Expected a positive decimal integer argument to -maxdepth, but got ‘-type’ This failure has been eliminated. 5.2.0.1 1) This release includes defect repair through Shorewall 5.1.12.4. 2) The getrc and getcaps commands added in 5.2.0 did not read the params file. That has been corrected. 3) A shell syntax error in the code that implements the 'ipdecimal' command has been corrected. 5.2.0 1) This release includes defect repair through Shorewall 5.1.12.3. 2) Previously, optimize category 8 (combine identical chains) was applied before optimize category 16 (eliminate duplicate rules, ...). This could (and has) resulted in uncombined identical chains in the final ruleset. Beginning with this release: a) Optimize category 16 will be applied before optimize category 8. b) If optimize category 8 combined any chains, then optimize category 16 will be applied again. This change ensures that the final ruleset has no duplicate chains and that all combatible adjacent port and state rules are combined. 3) Previously, use of &lo would result in an error: ERROR: Can't determine the IP address of lo: Firewall state not changed That problem has been corrected such that &lo always expands to 127.0.0.1 (IPv4) or ::1 (IPv6). ---------------------------------------------------------------------------- I I. K N O W N P R O B L E M S R E M A I N I N G ---------------------------------------------------------------------------- 1) On systems running Upstart, shorewall-init cannot reliably secure the firewall before interfaces are brought up. 2) The 'enable', 'reenable' and 'disable' commands do not work correctly in configurations with USE_DEFAULT_RT=No and optional providers listed in the DUPLICATE column. 3) While the 'ip' utility now accepts IPv6 routes with multiple 'nexthop' destinations, these routes are not balanced. They are rather instantiated as a sequence of single routes with different metrics. Furthermore, the 'ip route replace' command fails on such routes. Beginning with Shorewall6 5.0.15, the generated script uses a "delete..add.." sequence on these routes rather than a single "replace" command. ---------------------------------------------------------------------------- I I I. N E W F E A T U R E S I N T H I S R E L E A S E ---------------------------------------------------------------------------- 5.2.0.2 1) Răzvan Sandu has contributed a set of three macros for handling IPFS (see https://ipfs.io/). 5.2.0 1) The MAPOLDACTIONS option in shorewall.conf has been removed. This option provided compatibility with releases prior to Shorewall 3.0. 'shorewall update' will remove the setting of this option from shorewall.conf. 2) The INLINE_MATCH option has been removed. Shorewall now behaves as if INLINE_MATCH=No had been specified: - A single semicolon (';') is used to separate column-oriented input from column-name/value input. - The preferred method of specifying column-name/value input is to enclose such input in curly braces ("{....}"). - A pair of semicolons (';;') is used to introduce raw IP[6]TABLES input. This is true in INLINE and IP[6]TABLES rules as well as rules with other targets. As part of this change, 'shorewall update' will replace ';' with ';;' in INLINE and IP[6]TABLES rules. 3) With the wide availability of ipset-based blacklisting, the need for the 'refresh' command has been largely eliminated. As a result, that command has been removed. Some users may have been using 'refresh' as a lightweight form of reload. The most common of these uses seem to be for reloading traffic shaping after an interface has gone down and come back up. The best way to handle this situation under 5.2 is to make the interface 'optional' in your /etc/shorewall[6]/interfaces file, then either: - Install Shorewall-init and enable IFUPDOWN; or - Use the 'reenable' command when the interface comes back up in place of the 'refresh' command. 4) The following deprecated macros and actions have been removed: Action A_AllowICMPs - use AllowICMPs(A_ACCEPT) Action A_Drop - see below Action A_Reject - see below Action Drop - see below Action Reject - see below Macro SNMPTrap - use SNMPtrap The [A_]Drop and [A_]Reject actions are used primarily as policy actions. As part of this change, 'shorewall update' will update DROP_DEFAULT=[A_]Drop and REJECT_DEFAULT=[A_]Reject as follows: IPv4 DROP_DEFAULT=Drop becomes Broadcast(DROP),Multicast(DROP) DROP_DEFAULT=A_Drop becomes Broadcast(A_DROP),Multicast(A_DROP) REJECT_DEFAULT=Reject becomes Broadcast(DROP),Multicast(DROP) REJECT_DEFAULT=A_Reject becomes Broadcast(A_DROP),Multicast(A_DROP) IPv6 DROP_DEFAULT=Drop becomes AllowICMPs,Broadcast(DROP),Multicast(DROP) DROP_DEFAULT=A_Drop becomes AllowICMPs(A_ACCEPT),Broadcast(A_DROP),Multicast(A_DROP) REJECT_DEFAULT=Reject becomes AllowICMPs,Broadcast(DROP),Multicast(DROP) REJECT_DEFAULT=A_Reject becomes AllowICMPs(A_ACCEPT),Broadcast(A_DROP),Multicast(A_DROP) See the Migration Issues for additional information. 5) A 'show saves' command has been added to list the snapshots created using the 'save' command. Example: root@gateway:~# shorewall show saves Shorewall 5.2.0 Saves at gateway - Thu Feb 15 11:58:37 PST 2018 Saved snapshots are: Feb 15 10:08 foo Feb 14 12:34 restore (default) root@gateway:~# The snapshots are listed by creation time from latest to earliest. If the name of one matches the RESTOREFILE setting, that snapshot is marked as the default for the 'restore' command. 6) For installing into a Sandbox, the file shorewallrc.sandbox has been added to Shorewall-core. See http://www.shorewall.net/install.htm#idm327. 7) The "Use Pkttype Match (USEPKTTYPE)" capability is no longer used and has been deleted. This removal has introduced a new capabilities version. 8) When a log message is issued from a chain that relates to a pair of zones (e.g, 'fw-net'), the chain name normally appears in the log message (unless LOGTAGONLY=Yes and a log tag is specified). This can prevent OPTIMIZE category 8 from combining chains which are identical except for chain names in logging rules. The new LOG_ZONE option in shorewall[6].conf allows for only the source or destination zone to appear in the messages by setting LOG_ZONE to 'src' or 'dst' respectively. If LOG_ZONE=both (the default), then the full chain name is included in log messages Setting LOG_ZONE=src has been shown to decrease the size of the generated ruleset by more than 10 prcent in some cases. Your results may vary. 9) Traditionally, when OPTIMIZE category 8 is enabled, identical chains are combined under a name beginning with '~comb' or '~blacklist'. Beginning with this release, setting RENAME_COMBINED=Yes (the default) in shorewall[6].conf retains that behavior. If RENAME_COMBINED=No, identical chains are combined under the original name of one of the chains. 10) When AUTOMAKE=Yes, each directory in the CONFIG_PATH was originally searched recursively for files newer than the compiled script. That was changed in Shorewall 5.1.10.2 such that only the listed directories themselves were searched. That broke some configurations that played tricks with embedded SHELL such as: SHELL cat /etc/shorewall/rules.d/loc/*.rules Prior to 5.1.10.2, a change to a file in or adding a file to /etc/shorewall/rules.d/loc/ would trigger recompilation. Beginning with 5.1.10.2, such changes would not trigger recompilation. Beginning with this release, the pre-5.1.10.2 behavior can be obtained by setting AUTOMAKE=recursive. Also beginning with this release, AUTOMAKE may be set to a numeric which specifies how deeply each listed directory is to be searched. AUTOMAKE=1 only searches each directory itself and is equivalent to AUTOMAKE=Yes. AUTOMAKE=2 will search each directory and its immediate sub-directories; AUTOMAKE=3 will search each diretory, each of its immediate sub-directories, and each of their immediate sub-directories, etc. 11) Previously, the maximum depth of INCLUDEs was four (although the documentation gave the limit as three). Beginning with this release, that limit has been raised to 20. 12) Support for the deprecated 'masq' file has been deleted. Any existing 'masq' file will automatically be converted to the equivalent 'snat' file. 13) Three new shorewall commands have been implemented: a) show rc Displays the contents of the shorewallrc file ($SHAREDIR/shorewall/shorewallrc). b) getcaps Generates a capabilities file on a remote system and copies it to a directory on the local system. c) getrc Copies the shorewallrc file from a remote system to a directory on the local system. See shorewall(8) for details. Implemented by Matt Darfeuille ---------------------------------------------------------------------------- I V. M I G R A T I O N I S S U E S ---------------------------------------------------------------------------- If you are migrating from Shorewall 4.6.x or earlier, please see http://www.shorewall.net/pub/shorewall/5.0/shorewall-5.0.15/releasenotes.txt Immediately after installing Shorewall 5.2.x, we recommend that you run 'shorewall[6] update'. This command will handle many of the migration issues described here. ------------------------------------------------------------------------ I S S U E S M I G R A T I N G T O S H O R E W A L L 5 . 2 F R O M S H O R E W A L L 5 . 0 ------------------------------------------------------------------------ If you are migrating from Shorewall 5.0, this section will familiarize you with the changes in Shorewall 5.1 that may affect your configuration. 1) Shorewall 5.1 now has a single CLI program, ${SBINDIR}/shorewall (normally /sbin/shorewall). This program performs all of the same functions previously performed by /sbin/shorewall, /sbin/shorewall6, /sbin/shorewall-lite and /sbin/shorewall6-lite and is installed as part of the Shorewall-core package. It's default 'personality' is determined by the Shorewall packages installed: a) If the Shorewall package is installed, then by default, /sbin/shorewall behaves as in prior versions. b) If the Shorewall package is not installed, but the Shorewall-lite package is present, then /sbin/shorewall behaves as did /sbin/shorewall-lite in prior versions. c) If neither the Shorewall nor Shorewall-lite packages are installed, but the Shorewall6-lite package is installed, then /sbin/shorewall behaves as did /sbin/shorewall6-lite in prior versions. The program's personality can be altered through use of two new options. -6 When specified, changes the personality from Shorewall to Shorewall6 or from Shorewall-lite to Shorewall6-lite. -l When specified, changes the personality from Shorewall to Shorewall-lite or from Shorewall6 to Shorewall6-lite. This option is only required when both the standard package (Shorewall or Shorewall6) and the corresponding -lite package are installed on the system. The following is a comparison of Shorewall 5.0 and Shorewall 5.1 with respect to the CLI invocation: All four packages installed: Shorewall 5.0 Shorewall 5.1 shorewall shorewall shorewall6 shorewall -6 shorewall-lite shorewall -l shorewall6-lite shorewall -6l Only Shorewall-lite and Shorewall6-lite installed: Shorewall 5.0 Shorewall 5.1 shorewall-lite shorewall shorewall6-lite shorewall -6 A single shorewall(8) manpage now describes the CLI. The shorewall6(8), shorewall-lite(8) and shorewall6-lite(8) manpages are now minimal and refer the reader to shorewall(8). For backward compatibility, Shorewall6, Shorewall-lite and Shorewall6-lite install symlinks $SBINDIR/shorewall6, $SBINDIR/shorewall-lite and $SBINDIR/shorewall6-lite respectively. When the shorewall program is invoked through one of these symlinks, it adopts the appropriate personality. 2) The CHAIN_SCRIPTS option in the .conf files has been eliminated, and the compiler no longer looks for script files with the same name as a chain or action. If you are using such files, you will need to convert them into equivalent ?begin perl .... ?end perl text or to use the IP[6]TABLES target and/or inline matches. For the common case where you have an action xxx with an empty action.xxx file and have perl code in a file named xxx, the compiler will now generate a fatal error: ERROR: File action.xxx is empty and file xxx exists - the two must be combined as described in the Migration Considerations section of the Shorewall release notes For information about resolving this error, see http://www.shorewall.org/Shorewall-5.html#idp41228128. This issue is not handled by 'shorewall update' and must be corrected manually. 4) The Netfilter team have removed support for the rawpost table, so Shorewall no longer supports features requiring that table (stateless netmapping in the netmap file). The good news is that, since kernel 3.7, Netfilter supports stateful IPv6 network mapping which is now also supported in Shorewall6 (see shorewall6-netmap(5)). This issue is not handled by 'shorewall update' and must be corrected manually. 5) The (undocumented) Makefiles haven't been maintained for many releases and have been removed. 6) Beginning with Shorewall 5.1.2, The DROP_DEFAULT, REJECT_DEFAULT, etc. options may now specify a comma-separated list of actions rather than just a single action. The actions are invoked in the order in which they are listed and each action may optionally be followed by a colon (":") and a log level. The POLICY column in shorewall[6]-policy can now specify a similar list of actions. In that file, the list may be preceded by a plus sign ("+"), in which case the listed actions will be in addition to those listed in the related _DEFAULT setting in shorewall[6].conf. With these changes, the Drop and Reject policy actions are now deprecated in favor of a list of smaller actions. A warning is issued when these deprecated actions are used; the warning refers the reader to http://www.shorewall.net/Actions.html#Default. This issue is partially handled by 'shorewall update' - see the 5.2 issues below. 7) Beginning with Shorewall 5.1.2, the allowBcast, dropBcast, and Broadcast no longer handle multicast. Multicast is handeled separately in actions allowMcast, dropMcast and Multicast. The now-deprecated Drop and Reject policy actions have been modified so that they continue to silently drop multicast packets. 8) According to the Netfilter team (see https://patchwork.kernel.org/patch/9198133/), the --nflog-range option of the NFLOG target has never worked correctly, and they have deprecated that option in favor of the --nflog-size option. To accomodate this change, Shorewall 5.1.5 added an "--nflog-size support" (NFLOG_SIZE) Shorewall capability and a USE_NFLOG_SIZE option in shorewall[6].conf. If USE_NFLOG_SIZE=Yes, then if the capability is present, Shorewall will use '--nflog-size' in place of '--nflog-range'. If USE_NFLOG_SIZE=Yes and the capability is not present, an error is raised. If you don't use NFLOG or if you use NFLOG with omittted second parameter or with 0 as the second parameter, and 'shorewall show capabilities' indicated that --nflog-size support is present, you may safely set USE_NFLOG_SIZE=Yes. If you pass a non-zero value as the second parameter to NFLOG and the '--nflog-size support' capability is present, you need to verify that those NFLOG messages are as you expect with USE_NFLOG_SIZE=Yes. This issue is not handled by 'shorewall update' and must be corrected manually. 9) The MODULE_SUFFIX option in shorewall[6].conf was eliminated in Shorewall 5.1.7. Shorewall now finds modules, independent of their filename suffix. 'shorewall [-6] update' will automatically remove any MODULE_SUFFIX setting. 10) Beginning with Shorewall 5.1.8, when RESTORE_DEFAULT_ROUTE=Yes the default route is only restored when there are no enabled 'balance/primary' providers and no enabled fallback providers. Also beginning with Shorewall 5.1.8, if the default route(s) have been restored to the 'main' table, and a fallback provider is successfully enabled, the default route(s) are removed from the main table. 11) Because restoring default routes to the main routing table can break the ability of Foolsm and other link status monitors to properly detect non-functioning provider links, a warning message is issued when the 'persistent' provider option is specified and RESTORE_DEFAULT_ROUTE=Yes. WARNING: When RESTORE_DEFAULT_ROUTE=Yes, the 'persistent' option may not work as expected This change was released in Shorewall 5.1.8. This issue is not handled by 'shorewall update' and must be corrected manually. 12) Most interface OPTIONS have always been ignored when the INTERFACE name is '+'. Beginning with the Shorewall 5.1.10 release, a warning is issued when an ignored option is specified with interface name '+'. Example: The 'sourceroute' option is ignored when used with interface name '+' In many cases, this issue can be worked around by a change similar to the following: Original: net + dhcp,routeback,sourceroute=0 Change to: net all dhcp,physical=+,routeback,sourceroute=0 --- ---------- As part of this change, interfaces that specify a wildcard physical interface name will generate a warning if any of the following options are specified: accept_ra arp_filter arp_ignore forward logmartians proxyarp proxyndp routefilter sourceroute When the warning is issued, the specified option is then ignored for the interface. Example: WARNING: The 'sourceroute' option is ignored when used with a wildcard physical name /etc/shorewall6.universal/interfaces (line 14) This issue is not handled by 'shorewall update' and must be corrected manually. 13) INLINE_MATCHES=Yes has been documented as deprecated for some time, but it has not generated a warning. Beginning with the Shorewall 5.1.12 release, a warning is issued: WARNING: Option INLINE_MATCHES=Yes is deprecated Additionally, each line that requires modification to work with INLINE_MATCHES=No is flagged with the warning: WARNING: This entry needs to be changed (replace ';' with ';;') before the INLINE_MATCHES option is removed in Shorewall 5.2 You can eliminate the warnings by setting INLINE_MATCHES=No and by replacing the single semicolon (";") separating inline matches from the column-oriented part of the rule with two semicolons (";;") in each entry flagged by the second warning. This issue is mostly handled by 'shorewall update' - see the 5.2 issues below. ------------------------------------------------------------------------ I S S U E S M I G R A T I N G T O S H O R E W A L L 5 . 2 F R O M S H O R E W A L L 5 . 0 A N D 5 . 1 ------------------------------------------------------------------------ 1) The MAPOLDACTIONS option in shorewall.conf has been removed. This option provided compatibility with releases prior to Shorewall 3.0. 'shorewall update' will remove the setting of this option from shorewall.conf. 2) The INLINE_MATCH option has been removed. Shorewall now behaves as if INLINE_MATCH=No had been specified: - A single semicolon (';') is used to separate column-oriented input from column-name/value input. - The preferred method of specifying column-name/value input is to enclose such input in curly braces ("{....}"). - A pair of semicolons (';;') is used to introduce raw IP[6]TABLES input. This is true in INLINE and IP[6]TABLES rules as well as rules with other targets. As part of this change, 'shorewall update' will replace ';' with ';;' in INLINE and IP[6]TABLES rules. It will also replace ';' by ';;', if ';' is followed by '-m', '-j' or '-g'. 3) With the wide availability of ipset-based blacklisting, the need for the 'refresh' command has been largely eliminated. As a result, that command has been removed. Some users may have been using 'refresh' as a lightweight form of reload. The most common of these uses seem to be for reloading traffic shaping after an interface has gone down and come back up. The best way to handle this situation under 5.2 is to make the interface 'optional' in your /etc/shorewall[6]/interfaces file, then either: - Install Shorewall-init and enable IFUPDOWN; or - Use the 'reenable' command when the interface comes back up in place of the 'refresh' command. 4) The following deprecated macros and actions have been removed: Action A_AllowICMPs - use AllowICMPs(A_ACCEPT) Action A_Drop - see below Action A_Reject - see below Action Drop - see below Action Reject - see below Macro SNMPTrap - use SNMPtrap The [A_]Drop and [A_]Reject actions are used primarily as policy actions. As part of this change, 'shorewall update' will update DROP_DEFAULT=[A_]Drop and REJECT_DEFAULT=[A_]Reject as follows: IPv4 DROP_DEFAULT=Drop becomes Broadcast(DROP),Multicast(DROP) DROP_DEFAULT=A_Drop becomes Broadcast(A_DROP),Multicast(A_DROP) REJECT_DEFAULT=Reject becomes Broadcast(DROP),Multicast(DROP) REJECT_DEFAULT=A_Reject becomes Broadcast(A_DROP),Multicast(A_DROP) IPv6 DROP_DEFAULT=Drop becomes AllowICMPs,Broadcast(DROP),Multicast(DROP) DROP_DEFAULT=A_Drop becomes AllowICMPs(A_ACCEPT),Broadcast(A_DROP),Multicast(A_DROP) REJECT_DEFAULT=Reject becomes AllowICMPs,Broadcast(DROP),Multicast(DROP) REJECT_DEFAULT=A_Reject becomes AllowICMPs(A_ACCEPT),Broadcast(A_DROP),Multicast(A_DROP) The 'update' commmand will also make similar changes in the policy file. 'shorewall update' does not handle invocations of 'Drop' and 'Reject' within the rules file, or within actions and macros. Those instances will generate an error which must be corrected manually. It should also be noted that, in prior releases, Drop and Reject silently dropped more traffic than thir replacements. As a consequence, you will see more traffic being logged with Shorewall 5.2 than you did on earlier releases. The translations performed by 'update' can be extended after the update to drop additional traffic as desired. 5) When AUTOMAKE=Yes, each directory in the CONFIG_PATH was originally searched recursively for files newer than the compiled script. That was changed in Shorewall 5.1.10.2 such that only the listed directories themselves were searched. That broke some configurations that played tricks with embedded SHELL such as: SHELL cat /etc/shorewall/rules.d/loc/*.rules Prior to 5.1.10.2, a change to a file in or adding a file to /etc/shorewall/rules.d/loc/ would trigger recompilation. Beginning with 5.1.10.2, such changes would not trigger recompilation. Beginning with Shorewall 5.2.0, the pre-5.1.10.2 behavior can be obtained by setting AUTOMAKE=recursive. Also beginning with Shorewall 5.2.0, AUTOMAKE may be set to a numeric which specifies how deeply each listed directory is to be searched. AUTOMAKE=1 only searches each directory itself and is equivalent to AUTOMAKE=Yes. AUTOMAKE=2 will search each directory and its immediate sub-directories; AUTOMAKE=3 will search each diretory, each of its immediate sub-directories, and each of their immediate sub-directories, etc. 6) Support for the deprecated 'masq' file has been deleted. Any existing 'masq' file will automatically be converted to the equivalent 'snat' file. ---------------------------------------------------------------------------- V. N O T E S F R O M O T H E R 5 . 2 R E L E A S E S ----------------------------------------------------------------------------