E.45. Release 9.3.6

E.45.1. Migration to Version 9.3.6

A dump/restore is not required for those running 9.3.X.

However, if you are a Windows user and are using the "Norwegian (Bokmål)" locale, manual action is needed after the upgrade to replace any "Norwegian (Bokmål)_Norway" locale names stored in PostgreSQL system catalogs with the plain-ASCII alias "Norwegian_Norway". For details see http://wiki.postgresql.org/wiki/Changes_To_Norwegian_Locale

Also, if you are upgrading from a version earlier than 9.3.5, see Section E.46.

E.45.2. Changes

• Fix buffer overruns in to_char() (Bruce Momjian)

  When to_char() processes a numeric formatting template calling for a large number of digits, PostgreSQL would read past the end of a buffer. When processing a crafted timestamp formatting template, PostgreSQL would write past the end of a buffer. Either case could crash the server. We have not ruled out the possibility of attacks that lead to privilege escalation, though they seem unlikely. (CVE-2015-0241)

• Fix buffer overrun in replacement *printf() functions (Tom Lane)

  PostgreSQL includes a replacement implementation of printf and related functions. This code will overrun a stack buffer when formatting a floating point number (conversion specifiers e, E, f, F, g or G) with requested precision greater than about 500. This will crash the server, and we have not ruled out the possibility of attacks that lead to privilege escalation. A database user can trigger such a buffer overrun through the to_char() SQL function. While that is the only affected core PostgreSQL functionality, extension modules that use printf-family functions may be at risk as well.

  This issue primarily affects PostgreSQL on Windows. PostgreSQL uses the system implementation of these functions where adequate, which it is on other modern platforms. (CVE-2015-0242)

• Fix buffer overruns in contrib/pgcrypto (Marko Tiikkaja, Noah Misch)

  Errors in memory size tracking within the pgcrypto module permitted stack buffer overruns and improper dependence on the contents of uninitialized memory. The buffer overrun cases can crash the server, and we have not ruled out the possibility of attacks that lead to privilege escalation. (CVE-2015-0243)

• Fix possible loss of frontend/backend protocol synchronization after an error (Heikki Linnakangas)

  If any error occurred while the server was in the middle of reading a protocol message from the client, it could lose synchronization and incorrectly try to interpret part of the message's data as a new protocol message. An attacker able to submit crafted binary data within a command parameter might succeed in injecting his own SQL commands this way. Statement timeout and query cancellation are the most likely sources of errors triggering this scenario. Particularly vulnerable are applications that use a timeout and also submit arbitrary user-crafted data as binary query parameters. Disabling statement timeout will reduce, but not eliminate, the risk of exploit. Our thanks to Emil Lenngren for reporting this issue. (CVE-2015-0244)

• Fix information leak via constraint-violation error messages (Stephen Frost)

  Some server error messages show the values of columns that violate a constraint, such as a unique constraint. If the user does not have SELECT privilege on all columns of the table, this could mean exposing values that the user should not be able to see. Adjust the code so that values are displayed only when they came from the SQL command or could be selected by the user. (CVE-2014-8161)

• Lock down regression testing's temporary installations on Windows (Noah Misch)

  Use SSPI authentication to allow connections only from the OS user who launched the test suite. This closes on Windows the same vulnerability previously closed on other platforms, namely that other users might be able to connect to the test postmaster. (CVE-2014-0067)

• Cope with the Windows locale named "Norwegian (Bokmål)" (Heikki Linnakangas)

  Non-ASCII locale names are problematic since it's not clear what encoding they should be represented in. Map the troublesome locale name to a plain-ASCII alias, "Norwegian_Norway".

• Avoid possible data corruption if ALTER DATABASE SET TABLESPACE is used to move a database to a new tablespace and then shortly later move it back to its original tablespace (Tom Lane)

• Avoid corrupting tables when ANALYZE inside a transaction is rolled back (Andres Freund, Tom Lane, Michael Paquier)

  If the failing transaction had earlier removed the last index, rule, or trigger from the table, the table would be left in a corrupted state with the relevant pg_class flags not set though they should be.

• Ensure that unlogged tables are copied correctly during CREATE DATABASE or ALTER DATABASE SET TABLESPACE (Pavan Deolasee, Andres Freund)

• Fix incorrect processing of CreateEventTrigStmt.eventname (Petr Jelinek)

  This could result in misbehavior if CREATE EVENT TRIGGER were executed as a prepared query, or via extended query protocol.

• Fix DROP's dependency searching to correctly handle the case where a table column is recursively visited before its table (Petr Jelinek, Tom Lane)

  This case is only known to arise when an extension creates both a datatype and a table using that datatype. The faulty code might refuse a DROP EXTENSION unless CASCADE is specified, which should not be required.

• Fix use-of-already-freed-memory problem in EvalPlanQual processing (Tom Lane)

  In READ COMMITTED mode, queries that lock or update recently-updated rows could crash as a result of this bug.

• Avoid possible deadlock while trying to acquire tuple locks in EvalPlanQual processing (Álvaro Herrera, Mark Kirkwood)

• Fix failure to wait when a transaction tries to acquire a FOR NO KEY EXCLUSIVE tuple lock, while multiple other transactions currently hold FOR SHARE locks (Álvaro Herrera)

• Fix planning of SELECT FOR UPDATE when using a partial index on a child table (Kyotaro Horiguchi)

  In READ COMMITTED mode, SELECT FOR UPDATE must also recheck the partial index's WHERE condition when rechecking a recently-updated row to see if it still satisfies the query's WHERE condition. This requirement was missed if the index belonged to an inheritance child table, so that it was possible to incorrectly return rows that no longer satisfy the query condition.

• Fix corner case wherein SELECT FOR UPDATE could return a row twice, and possibly miss returning other rows (Tom Lane)

  In READ COMMITTED mode, a SELECT FOR UPDATE that is scanning an inheritance tree could incorrectly return a row from a prior child table instead of the one it should return from a later child table.

• Improve performance of EXPLAIN with large range tables (Tom Lane)

• Reject duplicate column names in the referenced-columns list of a FOREIGN KEY declaration (David Rowley)

  This restriction is per SQL standard. Previously we did not reject the case explicitly, but later on the code would fail with bizarre-looking errors.

• Re-enable error for SELECT ... OFFSET -1 (Tom Lane)

  A negative offset value has been an error since 8.4, but an optimization added in 9.3 accidentally turned the case into a no-op. Restore the expected behavior.

• Restore previous behavior of conversion of domains to JSON (Tom Lane)

  This change causes domains over numeric and boolean to be treated like their base types for purposes of conversion to JSON. It worked like that before 9.3.5 and 9.2.9, but was unintentionally changed while fixing a related problem.

• Fix json_agg() to not return extra trailing right brackets in its result (Tom Lane)

• Fix bugs in raising a numeric value to a large integral power (Tom Lane)

  The previous code could get a wrong answer, or consume excessive amounts of time and memory before realizing that the answer must overflow.

• In numeric_recv(), truncate away any fractional digits that would be hidden according to the value's dscale field (Tom Lane)

  A numeric value's display scale (dscale) should never be less than the number of nonzero fractional digits; but apparently there's at least one broken client application that transmits binary numeric values in which that's true. This leads to strange behavior since the extra digits are taken into account by arithmetic operations even though they aren't printed. The least risky fix seems to be to truncate away such "hidden" digits on receipt, so that the value is indeed what it prints as.

• Fix incorrect search for shortest-first regular expression matches (Tom Lane)

  Matching would often fail when the number of allowed iterations is limited by a ? quantifier or a bound expression.

• Reject out-of-range numeric timezone specifications (Tom Lane)

  Simple numeric timezone specifications exceeding +/- 168 hours (one week) would be accepted, but could then cause null-pointer dereference crashes in certain operations. There's no use-case for such large UTC offsets, so reject them.

• Fix bugs in tsquery @> tsquery operator (Heikki Linnakangas)

  Two different terms would be considered to match if they had the same CRC. Also, if the second operand had more terms than the first, it would be assumed not to be contained in the first; which is wrong since it might contain duplicate terms.

• Improve ispell dictionary's defenses against bad affix files (Tom Lane)

• Allow more than 64K phrases in a thesaurus dictionary (David Boutin)

  The previous coding could crash on an oversize dictionary, so this was deemed a back-patchable bug fix rather than a feature addition.

• Fix namespace handling in xpath() (Ali Akbar)

  Previously, the xml value resulting from an xpath() call would not have namespace declarations if the namespace declarations were attached to an ancestor element in the input xml value, rather than to the specific element being returned. Propagate the ancestral declaration so that the result is correct when considered in isolation.

• Ensure that whole-row variables expose nonempty column names to functions that pay attention to column names within composite arguments (Tom Lane)

  In some contexts, constructs like row_to_json(tab.*) may not produce the expected column names. This is fixed properly as of 9.4; in older branches, just ensure that we produce some nonempty name. (In some cases this will be the underlying table's column name rather than the query-assigned alias that should theoretically be visible.)

• Fix mishandling of system columns, particularly tableoid, in FDW queries (Etsuro Fujita)

• Fix assorted oversights in range-operator selectivity estimation (Emre Hasegeli)

  This patch fixes corner-case "unexpected operator NNNN" planner errors, and improves the selectivity estimates for some other cases.

• Avoid doing indexed_column = ANY (array) as an index qualifier if that leads to an inferior plan (Andrew Gierth)

  In some cases, = ANY conditions applied to non-first index columns would be done as index conditions even though it would be better to use them as simple filter conditions.

• Fix "variable not found in subplan target list" planner failure when an inline-able SQL function taking a composite argument is used in a LATERAL subselect and the composite argument is a lateral reference (Tom Lane)

• Fix planner problems with nested append relations, such as inherited tables within UNION ALL subqueries (Tom Lane)

• Fail cleanly when a GiST index tuple doesn't fit on a page, rather than going into infinite recursion (Andrew Gierth)

• Exempt tables that have per-table cost_limit and/or cost_delay settings from autovacuum's global cost balancing rules (Álvaro Herrera)

  The previous behavior resulted in basically ignoring these per-table settings, which was unintended. Now, a table having such settings will be vacuumed using those settings, independently of what is going on in other autovacuum workers. This may result in heavier total I/O load than before, so such settings should be re-examined for sanity.

• Avoid wholesale autovacuuming when autovacuum is nominally off (Tom Lane)

  Even when autovacuum is nominally off, we will still launch autovacuum worker processes to vacuum tables that are at risk of XID wraparound. However, such a worker process then proceeded to vacuum all tables in the target database, if they met the usual thresholds for autovacuuming. This is at best pretty unexpected; at worst it delays response to the wraparound threat. Fix it so that if autovacuum is turned off, workers only do anti-wraparound vacuums and not any other work.

• During crash recovery, ensure that unlogged relations are rewritten as empty and are synced to disk before recovery is considered complete (Abhijit Menon-Sen, Andres Freund)

  This prevents scenarios in which unlogged relations might contain garbage data following database crash recovery.

• Fix race condition between hot standby queries and replaying a full-page image (Heikki Linnakangas)

  This mistake could result in transient errors in queries being executed in hot standby.

• Fix several cases where recovery logic improperly ignored WAL records for COMMIT/ABORT PREPARED (Heikki Linnakangas)

  The most notable oversight was that recovery_target_xid could not be used to stop at a two-phase commit.

• Prevent latest WAL file from being archived a second time at completion of crash recovery (Fujii Masao)

• Avoid creating unnecessary .ready marker files for timeline history files (Fujii Masao)

• Fix possible null pointer dereference when an empty prepared statement is used and the log_statement setting is mod or ddl (Fujii Masao)

• Change "pgstat wait timeout" warning message to be LOG level, and rephrase it to be more understandable (Tom Lane)

  This message was originally thought to be essentially a can't-happen case, but it occurs often enough on our slower buildfarm members to be a nuisance. Reduce it to LOG level, and expend a bit more effort on the wording: it now reads "using stale statistics instead of current ones because stats collector is not responding".

• Fix possible corruption of postmaster's list of dynamic background workers (Andres Freund)

• Fix SPARC spinlock implementation to ensure correctness if the CPU is being run in a non-TSO coherency mode, as some non-Solaris kernels do (Andres Freund)

• Warn if macOS's setlocale() starts an unwanted extra thread inside the postmaster (Noah Misch)

• Fix processing of repeated dbname parameters in PQconnectdbParams() (Alex Shulgin)

  Unexpected behavior ensued if the first occurrence of dbname contained a connection string or URI to be expanded.

• Ensure that libpq reports a suitable error message on unexpected socket EOF (Marko Tiikkaja, Tom Lane)

  Depending on kernel behavior, libpq might return an empty error string rather than something useful when the server unexpectedly closed the socket.

• Clear any old error message during PQreset() (Heikki Linnakangas)

  If PQreset() is called repeatedly, and the connection cannot be re-established, error messages from the failed connection attempts kept accumulating in the PGconn's error string.

• Properly handle out-of-memory conditions while parsing connection options in libpq (Alex Shulgin, Heikki Linnakangas)

• Fix array overrun in ecpg's version of ParseDateTime() (Michael Paquier)

• In initdb, give a clearer error message if a password file is specified but is empty (Mats Erik Andersson)

• Fix psql's \s command to work nicely with libedit, and add pager support (Stepan Rutz, Tom Lane)

  When using libedit rather than readline, \s printed the command history in a fairly unreadable encoded format, and on recent libedit versions might fail altogether. Fix that by printing the history ourselves rather than having the library do it. A pleasant side-effect is that the pager is used if appropriate.

  This patch also fixes a bug that caused newline encoding to be applied inconsistently when saving the command history with libedit. Multiline history entries written by older psql versions will be read cleanly with this patch, but perhaps not vice versa, depending on the exact libedit versions involved.

• Improve consistency of parsing of psql's special variables (Tom Lane)

  Allow variant spellings of on and off (such as 1/0) for ECHO_HIDDEN and ON_ERROR_ROLLBACK. Report a warning for unrecognized values for COMP_KEYWORD_CASE, ECHO, ECHO_HIDDEN, HISTCONTROL, ON_ERROR_ROLLBACK, and VERBOSITY. Recognize all values for all these variables case-insensitively; previously there was a mishmash of case-sensitive and case-insensitive behaviors.

• Make psql's \watch command display nulls as specified by \pset null (Fujii Masao)

• Fix psql's expanded-mode display to work consistently when using border = 3 and linestyle = ascii or unicode (Stephen Frost)

• Fix pg_dump to handle comments on event triggers without failing (Tom Lane)

• Allow parallel pg_dump to use --serializable-deferrable (Kevin Grittner)

• Improve performance of pg_dump when the database contains many instances of multiple dependency paths between the same two objects (Tom Lane)

• Fix pg_dumpall to restore its ability to dump from pre-8.1 servers (Gilles Darold)

• Fix possible deadlock during parallel restore of a schema-only dump (Robert Haas, Tom Lane)

• Fix core dump in pg_dump --binary-upgrade on zero-column composite type (Rushabh Lathia)

• Fix failure to fsync tables in nondefault tablespaces during pg_upgrade (Abhijit Menon-Sen, Andres Freund)

  With an operating system crash and some bad luck, this could result in data loss during an upgrade.

• In pg_upgrade, cope with cases where the new cluster creates a TOAST table for a table that didn't previously have one (Bruce Momjian)

  Previously this could result in failures due to OID conflicts.

• In pg_upgrade, don't try to set autovacuum_multixact_freeze_max_age for the old cluster (Bruce Momjian)

  This could result in failure because not all 9.3.X versions have that parameter. Fortunately, we don't actually need to set it at all.

• In pg_upgrade, preserve the transaction ID epoch (Bruce Momjian)

  This oversight did not bother PostgreSQL proper, but could confuse some external replication tools.

• Prevent WAL files created by pg_basebackup -x/-X from being archived again when the standby is promoted (Andres Freund)

• Fix memory leak in pg_receivexlog (Fujii Masao)

• Fix unintended suppression of pg_receivexlog verbose messages (Fujii Masao)

• Fix failure of contrib/auto_explain to print per-node timing information when doing EXPLAIN ANALYZE (Tom Lane)

• Fix upgrade-from-unpackaged script for contrib/citext (Tom Lane)

• Avoid integer overflow and buffer overrun in contrib/hstore's hstore_to_json() (Heikki Linnakangas)

• Fix recognition of numbers in hstore_to_json_loose(), so that JSON numbers and strings are correctly distinguished (Andrew Dunstan)

• Fix block number checking in contrib/pageinspect's get_raw_page() (Tom Lane)

  The incorrect checking logic could prevent access to some pages in non-main relation forks.

• Fix contrib/pgcrypto's pgp_sym_decrypt() to not fail on messages whose length is 6 less than a power of 2 (Marko Tiikkaja)

• Fix file descriptor leak in contrib/pg_test_fsync (Jeff Janes)

  This could cause failure to remove temporary files on Windows.

• Handle unexpected query results, especially NULLs, safely in contrib/tablefunc's connectby() (Michael Paquier)

  connectby() previously crashed if it encountered a NULL key value. It now prints that row but doesn't recurse further.

• Avoid a possible crash in contrib/xml2's xslt_process() (Mark Simonetti)

  libxslt seems to have an undocumented dependency on the order in which resources are freed; reorder our calls to avoid a crash.

• Mark some contrib I/O functions with correct volatility properties (Tom Lane)

  The previous over-conservative marking was immaterial in normal use, but could cause optimization problems or rejection of valid index expression definitions. Since the consequences are not large, we've just adjusted the function definitions in the extension modules' scripts, without changing version numbers.

• Numerous cleanups of warnings from Coverity static code analyzer (Andres Freund, Tatsuo Ishii, Marko Kreen, Tom Lane, Michael Paquier)

  These changes are mostly cosmetic but in some cases fix corner-case bugs, for example a crash rather than a proper error report after an out-of-memory failure. None are believed to represent security issues.

• Fix setup of background workers in EXEC_BACKEND builds, eg Windows (Robert Haas)

• Detect incompatible OpenLDAP versions during build (Noah Misch)

  With OpenLDAP versions 2.4.24 through 2.4.31, inclusive, PostgreSQL backends can crash at exit. Raise a warning during configure based on the compile-time OpenLDAP version number, and test the crashing scenario in the contrib/dblink regression test.

• In non-MSVC Windows builds, ensure libpq.dll is installed with execute permissions (Noah Misch)

• Make pg_regress remove any temporary installation it created upon successful exit (Tom Lane)

  This results in a very substantial reduction in disk space usage during make check-world, since that sequence involves creation of numerous temporary installations.

• Support time zone abbreviations that change UTC offset from time to time (Tom Lane)

  Previously, PostgreSQL assumed that the UTC offset associated with a time zone abbreviation (such as EST) never changes in the usage of any particular locale. However this assumption fails in the real world, so introduce the ability for a zone abbreviation to represent a UTC offset that sometimes changes. Update the zone abbreviation definition files to make use of this feature in timezone locales that have changed the UTC offset of their abbreviations since 1970 (according to the IANA timezone database). In such timezones, PostgreSQL will now associate the correct UTC offset with the abbreviation depending on the given date.

• Update time zone abbreviations lists (Tom Lane)

  Add CST (China Standard Time) to our lists. Remove references to ADT as "Arabia Daylight Time", an abbreviation that's been out of use since 2007; therefore, claiming there is a conflict with "Atlantic Daylight Time" doesn't seem especially helpful. Fix entirely incorrect GMT offsets for CKT (Cook Islands), FJT, and FJST (Fiji); we didn't even have them on the proper side of the date line.

• Update time zone data files to tzdata release 2015a.

  The IANA timezone database has adopted abbreviations of the form AxST/AxDT for all Australian time zones, reflecting what they believe to be current majority practice Down Under. These names do not conflict with usage elsewhere (other than ACST for Acre Summer Time, which has been in disuse since 1994). Accordingly, adopt these names into our "Default" timezone abbreviation set. The "Australia" abbreviation set now contains only CST, EAST, EST, SAST, SAT, and WST, all of which are thought to be mostly historical usage. Note that SAST has also been changed to be South Africa Standard Time in the "Default" abbreviation set.

  Also, add zone abbreviations SRET (Asia/Srednekolymsk) and XJT (Asia/Urumqi), and use WSST/WSDT for western Samoa. Also, there were DST law changes in Chile, Mexico, the Turks & Caicos Islands (America/Grand_Turk), and Fiji. There is a new zone Pacific/Bougainville for portions of Papua New Guinea. Also, numerous corrections for historical (pre-1970) time zone data.