lfcbase:
- Completed integer cleanup for Pager class,
replacement of some asInteger with asUnsigned method calls
( BigInteger, NetHandler, etc. )
cego:
- Added several patches for trigger objects
( check self refering tables in trigger block to avoid recursion,
object invalidation handling fixed )
- In CegoRecoveryManager::recoverCurrentTransactionLog,
completed recovery procedure for trigger and alias objects.
For drop operations, added CegoDatabaseManager::removeObject calls,
to remove corresponding objects from object cache
- integer cleanup, all int values will be replaced by unsigned
or unsigned long, where is no need of nagative values
- for those ports:
devel/lfcbase
devel/lfcxml
databases/cego
databases/cegobridge
databases/p5-DBD-cego
cego:
- Fix added in CegoDbHander::acceptSession and CegoSerial::readChain
to treat empty or invalid session requests
- fix in CegoDbThreadPool for bad message handling
p5-DBD-cego:
- Fix added for execute return value and $sth->rows value to
return affected rows for update statements
lfcbase:
- Warning elimination in several code modules.
-Wall should print no warnings anymore (with clang)
cego:
- Added patch for CegoJoinObject for correct and complete decoding.
This is needed for database recovery procedure if nested join select
occurs in update statement, e.g.
update t1 set b1 = ( select max(b2) from t2 inner join t3 on t2.a2 = t3.a3 );
- Warning elimination for several code modules.
In CegoContentObject encoding and decoding method,
the obsolete key flag in the schema information was removed.
This leads to a incompatibel datastructure and requires tableset
export/import for an uplift ot this version
- To toChain method of CegoTableObject, CegoJoinObject and
CegoAliasObject has been improved to treat undefined tabSetId
argument ( defTabSetId = 0 ).
lfcbase:
- Added patch in File::hasData to check return code of pool library call
- Added fix in basecheck ReplaceTest to return valid results
cego:
- For CegoClient and CegoAdmin changed argument evaluation to handle
pipe input data in a better way.
Pipe mode is just used, if no batch or command mode ist setup and
if data is available at stdin.
- This should also work for FreeBSD, when cgclt or cgadm is used
via cronjob. It has been observed, that if a program is called by
cron, poll returns data available at stdin also if there is no data
available.
- In CegoAdminThread, for tableset and table import procedure added
switch to next backup branch.
This is required, since during import, logging is disabled.
So after import, a new backup is needed.
A new uses 'sudo' has been added to transperently depends of the
proper flavor of sudo 'default' or 'sssd' depending on the default
version set by the user.
If you prefere to use 'sssd' flavor of sudo, add the following
to your make.conf:
DEFAULT_VERSIONS+=sudo=sssd
All sudo consummer ports has benn updated to USES=sudo
Reviewed by: 0mp
Approved by: mat
Sponsored by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D52160
NOTE: Since this patch changed content object encoding,
the upgrade requires a tableset export/import !
- Fix added for CegoLogReader. __dateTimeFormat has to be setup
to print date values in log data
- In CegoContentObject, encoding and decoding was not done for tableAlias.
This might lead query errors during crash or backup recovery for insert
or update statements with nested selects.
Table Alias has been added to encoding.
- Fix added in CegoAggration decoding constructor.
_pExpr must be set to 0, otherwise this might result in seg fault
during tableset recovery.
- Added stdin mode for cego admin console cgadm, so instead
of using option --batchfile, admin commands can be feed by pipe
- Cleanup and completion work done for CegoLogReader ( cglog ).
For this also the abstract method CegoDecodable::getFormatted
has been changed to getInfo, which provides a more compact output format.
cglog is used to print ( binary ) redo logfiles in a human readable form.
- For CegoLogReader added stdin feature, so log file can be piped
via stdin to cglog
- Extended backup list to indicate most recent redo log timestamp.
For this, the backupManager utility had to be adapted to retrieve
last redo log timestamp from arch and tape directory
- Added CegoXMLSpace::adjustPageOffset method to adjust page offset
attribute after tableset recovery. This is needed, if a recovery
is made from scratch with a new initialized database xml file
- Changed method CegoXMLSpace::nextTSID to generic.
Instead of calculating the next tableset id on XML_MAXTSID_ATTR,
the id is calculated based on all defined tablesets.
In case of a from scratch recovery ( with a new initialized db xml file ),
this ensures id consistency
- improved utility scripts for online backup to tools directory
( logManager, backupManager, backupExecuter )
- more stabilzation for online backup recovery from scratch,
e.g. checking for existing tableset set entry in db xml during recovery
and allow non existing entry.
Perhaps it might be useful to check pagesize of backup datafiles
and defined pagesize in db xml ...
- added tableset define parameter branchid.
This parameter can be used to set up an appropriate backup branchid
for from scratch recovery procedures
- patch added for scratch recovery.
If a tableset is defined for backup recovery, the tableset id
must be adjusted to the corresponding id in the datafile.
For this, new methods CegoTableManager::adjustTableSetId and
CegoFileHandler::readTableSetId have been introduced and added
to CegoRecoveryManager::recoverTableSet
- In CegoRecoveryManager::recoverTableSet: shifted ticket recovery
before file registration.
So all needed tableset id values are set up correctly and there's
no more need to adjust any ids.
Just root path and branch id must contain appropriate values
for newly defined tableset to be recovered
- Fix added in CeegoXMLSpace::getTableSetBackupBranch,
removed a double V() operation which caused XML locking issue
- Changed BRANCHID attribute to NEXTBRANCHID in XML root tag,
to clarify the meaning.
- Changed Backup Branch Id from timestamp value to
database wide unique numeric id.
For this CegoXMLSpace::setTableSetBackupBranch method
has changed to nextTableSetBackupBranch
- Block variables in CegoDistCursor still have not been handled
- For this, the constructor has been expanded to take a block variable
as an argument. The block variable then is propagated to the
corresponding eval methods ( of CegoPredicate objects )
- check124 has been added to illustrate a case, where this patch is needed.
- Improved input handling in CegoClient.
For interactive mode, trailing LF, CR and multiple semicolons are cut now.
- Added command line option lockexpire with integer parameter seconds
exceeded.
If set, this option will check for the configured daemon lockfile
and if the timestamp of the lockfile is older than now - seconds
exceeded, the lock file is removed and the database is started.
Database crash recovery seems to be stable, so this option is made available.
- In CegoDbThread::serveRequest, added further tableset states (
BACKUP, CHECKPOINT ) where sessions are accepted
- fix performance for CegoPredicate
- CLI: threadinfo db, lastAction was added to the threadlist to
indicate the query currently processing
- format improvements for CegoAction::getPlanString to indent subqueries
- handle join master predicate in the most optimal way.
To show several condition cases, check123.sql has been added to
demonstrate the join optimization for different join cases
- fixed some cases for master predicate handling
example:
plan select cpid from companyview cv where lower(cv.cpname) like '%%'
or debnr = 3;
shows the execution plan for the query.
lfcbase:
- Fixed BigDecimal class for internal constructor to
normalize zero values to "0" and in scaleTo method to treat
zero values correctly
- Added class Replacer to support string replacement based
on regular expressions ( like unix sed )
- Fixed BigDecimal, scale handling still not correct for some cases
- Replacer: The match token has been changed to $ char to treat
database integration
cego:
- Performance optimization added in CegoSelect and CegoDistCursor
for or-condition handling:
If an or-condition is recognized, the predicate is propagateed
to the corresponding join level cursor and the evaluation is handled
by the cursor.
This can avoid large table cursor result sets for the embedding
join and might improve overall performance of the query.
- For the new introduced query optimization, several code extensions
have been added. In CegoPredicate, new methods map, getReduced,
mapExpr and reduceExpr have been added to convert the "master"
predicate to an appropriate form, which can be handled by CegoDistCursor.
Also for for CegoDistCursor, the distSetup methods have been expanded
to propagate the master predicate
- Added fix in CegoAction for the following query:
select invid from invoice where status = ( select 'OPEN' );
In CegoAction::noFromOption, for _coListStack has to be pushed empty coList
- Added verify010 db check with a small invoice er model. This is
used for advanced query verification with multilevel nested views
and other dynamic stuff which is also relevant to proof valid results
from query cache
lfcbase:
- BigDecimal::scaleTo: added RoundMode parameter which can either be
UP ( round up )
DOWN ( round down )
HALFUP ( round up if neigbours equidistant or higher ) or
HALFDOWN ( rond down if neighbors are equidistant or lower )
cego:
- Added support for lfcbase-1.21.8 ( BigDecimal::RoundMode ),
Round mode ist setup in dbxml via database ROUNDMODE attribute
( either UP, DOWN, HALFUP, HALFDOWN )
- Added performance patch to CegoSelect::buildJoinConditions. pPred
is set to checked if analyzed as complete, also if the predicate
is external ( setup via setViewCond ) In the meantime, external
conditions are fully handeled via CegoDistCursor
- In CegoFieldValue::toChain, for case DATETIME resulting string
was corrected. In the meantime, we avoid date value construction
via date/scandate function, either the required value is calculated
via cast. So just the date string value is provided in the defined
datetime format.
- Added datetime cast operator to convert from string value to
datetime. If used, this avoids expensive real time cast operations
for queries and should accelerate execution time
- Fix added in CegoQueyHelper::evalAttrCond, COMPLETE value must
be returned in flArray loop and if not found PARTIAL must be returned
( return was missing )
- Patch in CegoTerm::getSelectQueryList to retrieve subselects
also for case conditions and nested conditions. So this affects
also class CegoFactor, CegoCaseCond and CegoCondition which have
to provide the getSelectQueryList method. This extension is required
to retrieve appropriate table sets for query cache decicisions
- Added tableset online check in CegoDbThread::serveRequest.
This avoids query execution of any connected client during recovery phase
- Improved log trace in CegoDistManager::startTableSet to log complete
stack trace in case of tableset startup failure
- In CegoAdminHandler::getUserInfo, increased size
of NUMREQ and NUMQUERY from 10 to 12, for long running systems
with > 10^9 requests
- Added pack mode in CegoClient to improve the display of
large string column values. Only the longest length of all values
in <n> rows is displayed, not the full length of the field itself.
Example: If the table definition for a string is 300 chars,
and the values are only 4-10 chars wide, the table is only
displayed with 10 chars.
- Fix added in CegoDistCursor::joinSetup:
for outerCond calculation, null value comparisons are also used,
since this leads to correct join results and can be optimized by
using btrees.
( set attrCond.getFilterCond(outerSchema, false) to (outerSchema, true)
for left outer and right outer join)
- In CegoSelect::getPlan, added parentJoinBuf setup for nested queries in expression list to evaluate join plan correctly
( parentJoinBuf is required for analysis in CegoQueryHelper::checkAttrCond)
- Add duplicate check for table aliases in CegoSelect::evalReferences
- Fix in CegoSelect constructors to initialize _joinLevel to 0.
A problem may arise, if the reset method is called without
a previous select analyse which happens in nested subselects
lfcbase:
- Added improvements to Pager class, e.g. case insensitive search and
message box at end of search
cego:
- Small adaptions made for Debian
( in cgwtest.c, cast added to malloc for CGClob and CGBlob allocation ).
There are still required some changes for the testsuite shell scripts
to satisfy Debian flavour ( expimpcheck )
- Patch added in CegoCondition and CegoPredicate for arithmetic
expression evaluation. The recent grammar modification limits
enclosed arithmetic expression to logic expressions. This leads
confusing results e.g. for
select ( a + b ) / 2 from t1 since ( a + b )
is evaluated as a condition with a logical value as the result
( true or false )
( caused by grammar production : Factor -> LB Condition RB )
Arithmetic expressions are just parsed with grammar production
Predicate -> Expr ( expression only ), so for this case now a
special handling has been added
( with new introduced method CegoPredicate::getExprOnlyValue )
lfcbase:
- To support several keyboard layouts, changed some window control
sequences from Cntrl-Key to ESC key
- Added virtual method hasData to Pager class. The method must be
implemented to indicate, if an execution returns a result set. If
no result set is returned, just a message box is shown and edit
mode is left ). This seems to be useful for DML queries ( insert,
udate, delete, create, drop, ...)
- In Pager class, added buffer scrolling in editor part. Preparation
started for Pager command history management
- Added history feature to Pager class. Command history can be loaded
and saved with readHistory and writeHistory methods.
Navigation through command history in Edit mode with Cntrl-P and Cntrl-N
cego:
- Adaptions made for pager and pager history feature
lfcbase:
- more patches for the Pager class
Removed init_color with custom color for select cursor since the
custom color seems to be poor supported for curses. Fixes added
to Pager::writeFooter
- edit-feature in Pager (must discuss this with Bjoern 8-)
- ignore SIGINT in Pager
- change from unsigned long long to long long value in Datetime class
- pattern search feature for Pager class
cego:
- Small fix in CegoBufferPool.h
- CegoClient adapted to new Pager API
- Queries now can be interrupted with Cntrl-C also in pager mode
- adapt to Datetime long long values ( e.g. date2str from int to long long )
lfcbase:
- Rework of Pager class, which is a API change
cego:
- Made some adaptions for reworked lfcbase Pager class
rows with multiline entries are more readable
- Added check script check119.sql to verify grammar expansion
for stored procedure condition
- CegoAction::procArg1() and CegoAction::procArg2() to check
duplicate parameter names
- Added check118.sql to demonstrate explicit table locking for nested
delete operation within a cursor loop
- Fix added in CegoFactor::getAggregationList:
for case QUERY and case FETCH, the not-supported exceptions
have been removed, otherwise this leads a useless exception
in CegoAttrCond::asConjunctionList where aggregations are just checked
- Added grammer change in Cego.def to ensure operator precedence
for AND and OR condition operator.
AND has higher precedence if no enclosing braces are given.
The result for the following queries should be the same
select * from t1 where a = 42 or a = 1 and b = 'X' ;
select * from t1 where a = 1 and b = 'X' or a = 42;
select * from t1 where a = 42 or ( a = 1 and b = 'X' );
- With cego version before 2.51.7, braces have been mandatory to
ensure operator precedence.
With the new grammar extension, the AND operator has a higher presedence
- CegoFactor::evalFieldValue, QUERY case, after query
evaluation, the parentJoinBuf has to be reset, otherwise an invalid
reference is used later to CegoSelect::getQueryId. This might result
in core dumps, if query cache is enabled
- Added querycache verification to db check routines
- Added querycache mode to command line options to set up query cache by batch
lfcbase:
- BigDecimal::div null value check
cego:
- CegoFactor::toChain to add braces for condition case
- CegoSelect::initAggregation, for sum and avg aggregation, aggregation
value is initialized with 0 and appropriate datatype ( instead of
null value ). This leads to numeric result 0 in case of empty result
sets ( instead of null value )
- Fix made in CegoDistCursor::distSetup to support FLA setting with
no cursor condition. This is needed for query plan evaluation for
nested inner/left outer/right outer joins
See check114 as a sample
devel/lfcbase 1.18.11 -> 1.19.0
- Changed Datetime::asLong to return signed long values ( long long ).
This is needed to support timestamp values before 01.01.1970,
which are signed negative
databases/cego 2.50.6 -> 2.51.0
- consolidate grammar productions for query conditions and
procedure conditions
- Since CegoPredicate::CompType has received new type EXPRONLY,
this requires a tableset export/import for upgrading to this version
( for predicate encoding / decoding )
- Several changed in CegoFieldValue to switch from Datetime integer
type unsigned long long to long long. This is required to support
datetime comparison for values before 01.01.1970
databases/cegobridge 1.8.1 -> 1.9.0
- adapt to cego 2.51.0
- Optimization added to CegoPredDesc::match. Matcher is just created,
if the match method is called, since it might occur, that the
condition is evaluated be CegoDistCursor via CegoAttrCond
- Fix in CegoAttrComp::getMatcher, there was still missing star
escape, which has been added to CegoPredDesc constructor. To avoid
two same pieces of code, CegoQueryHelper::sql2Regex has been
introduced
cego:
- Added syntax expansion for group having clause to support
advanced having conditions, e.g.
SELECT a, sum(b), max(c) FROM t1
GROUP BY a HAVING sum(b) > 10 AND max(c) = 100;
- Patch added for 'view condition evaluation' in
CegoAttrCond::asConjunctionList, CegoSelect::setViewCond and
CegoDistCursor::distSetup.
- Additional check added to CegoObjectManager to avoid duplicate
table objects of different type ( e.g. Table t1 and View t1 ). Table
objects are either native tables, views or aliases.
- For predicate export structure, changed from XML representation
to native SQL syntax. This requires a dedicated Parser in CegoXPorter,
which can handle predicate SQL representation, but saves lots of
code, since all subsequent member classes of CegoPredDesc no more
need to implement XML representation methods ( constructor,
fromElement, toElement )
- Added stabilization patches for nested grouping select, e.g. in
exists subselect. In CegoSelect::reset, for _selectMode == GROUPING,
grouping space is resetted now properly.
- Added stabilization patches to CegoAction. Load statements for
views, procedures, triggers and condition are treated regarding
object allocation/free. These statements are for internal use only,
so a database user receives a note message now
- Further improvements added in CegoQuery to give more detailed
error information in case of exceptions
- In CegoTableManager, made some exception message improvements,
instead of popping last message, the complete message stack is thrown
cegobridge:
- Support added for cego-2.50.x
- Fixes added for modified CegoAction::getSelect and
CegoAction::getProcedure methods
- For several modules, added exception trace information to improve
error logging in case of any exception
- Patch added for CegoQueryCache::addEntry, getNumUsed added to
search loop to find appropriate slot. This avoids some needless
assignments, since used entries cannot be assigned.
- Patch added CegoQueryCache::invalidate. For main loop, a delay
has been added, to give other threads the chance to claim the lock
- Patch for CegoFunction added.
For methods getReturnType, getReturnTypeLen and evelFieldValue
the following has been added : If proc object is getting used by
useObject, the following getProcedure call must be catched for
exception to unuse the object since _pProc is stiil null and the
object so is not unused by cleanup
- Removed lockPool call in CegoAdminThread::srvExportTableSet and
srvExportTable to avoid lock timeouts during long running exports
- Made some code cleanup in CegoAdminThread ( exception consolidation )
- db thread query history information, added timestamp and cost information
- Introduces new cgadm admin commands
o queryhist last n
o queryhist cost n
to show the query history for all db threads in the db thread
pool. The last n queries or the most expensive n queries are
indicated
- Added query history suppport for cgadm screen mode
- CegoAdmScreen to fully support of query history feature
( last and full over all db threads ).
- Screen2:Grid schema patches added to satisfy new column alignment
feature for Screen2
- Fix added in CegoDBThreadPool. CegoAdminThreadPool and CegoLogThreadPool.
If database hostname is not defined, the corresponding service is served
on any hostname. This avoids confusion regarding IPv4 and IPv6 host
configuration ( e.g. on MacOSX Sequoia, java seems to
handle localhost always as IPv4 127.0.0.1, while on POSIX layer,
it is mapped to ::1, a bit strange )
- Added new lfcbase Net::serve6 method to db, admin and log threadpool.
If no hostname is defined in database xml, an IP dual stack
configuration is set up.
- cgmkdb utility script:
allows to set up advanced database parameters
- In CegoXMLSpace::addTableSeDef, added ARCHMODE attribute to
set archmode to OFF as default
- Added querycache threshold parameter. Just queries with execution
time greater than threshold value are cached. This leads to a lower
stressed cache and avoids cache latency for those queries. Threshold
is given in msec, which seems to be appropriate for now.
- Some memory leaks fixed
- fix cgadm (hang if INTR received while idle)
- Added query history for db threads. Not just the last action is
indicated via admin command "threadinfo db threadId", but also the
last n queries processed by this thread. History Length can be
configured via new cego command line option dbthreadhist
- Patch added for memory leak in CegoSelect::cleanUp and
CegoSelect::reset, for both methods, _pCacheList has to checked and
freed, since for uncompleted queries ( e.g. exists clause ) the
_pCacheList was still not deleted
- Another fix for memory leak added in CegoFieldValue::getLocalCopy,
Before allocation, _len has to be checked for _len < STATICFIELDBUF
and then use _staticBuf, otherwise, the memory is lost
lfcbase:
- patch to Screen2::Form::handleKey to treat pipe character ( | )
- patch to Screen2::Attribute::setAttrList for refresh window
after attrlist has been updated
- Change in Screen2::Attribute class.
The class now provides three layout modes : HORIZONTAL, VERTICAL, OVERLAY.
In OVERLAY mode, with left and right cursor key it is switched
to visible attribute list. For this, the Screen2::Attribute constructor
has been changed
cego:
- For CegoAdmScreen, added QueryCacheInfo to tableset menu
- daptions made for CegoAdmScreen to
handle lfcbase-1.18.8 API change for Screen2::Attribute
