- Added tableset online check in CegoDbThread::serveRequest.
This avoids query execution of any connected client during recovery phase
- Improved log trace in CegoDistManager::startTableSet to log complete
stack trace in case of tableset startup failure
- In CegoAdminHandler::getUserInfo, increased size
of NUMREQ and NUMQUERY from 10 to 12, for long running systems
with > 10^9 requests
- Added pack mode in CegoClient to improve the display of
large string column values. Only the longest length of all values
in <n> rows is displayed, not the full length of the field itself.
Example: If the table definition for a string is 300 chars,
and the values are only 4-10 chars wide, the table is only
displayed with 10 chars.
- Fix added in CegoDistCursor::joinSetup:
for outerCond calculation, null value comparisons are also used,
since this leads to correct join results and can be optimized by
using btrees.
( set attrCond.getFilterCond(outerSchema, false) to (outerSchema, true)
for left outer and right outer join)
- In CegoSelect::getPlan, added parentJoinBuf setup for nested queries in expression list to evaluate join plan correctly
( parentJoinBuf is required for analysis in CegoQueryHelper::checkAttrCond)
- Add duplicate check for table aliases in CegoSelect::evalReferences
- Fix in CegoSelect constructors to initialize _joinLevel to 0.
A problem may arise, if the reset method is called without
a previous select analyse which happens in nested subselects
lfcbase:
- Added improvements to Pager class, e.g. case insensitive search and
message box at end of search
cego:
- Small adaptions made for Debian
( in cgwtest.c, cast added to malloc for CGClob and CGBlob allocation ).
There are still required some changes for the testsuite shell scripts
to satisfy Debian flavour ( expimpcheck )
- Patch added in CegoCondition and CegoPredicate for arithmetic
expression evaluation. The recent grammar modification limits
enclosed arithmetic expression to logic expressions. This leads
confusing results e.g. for
select ( a + b ) / 2 from t1 since ( a + b )
is evaluated as a condition with a logical value as the result
( true or false )
( caused by grammar production : Factor -> LB Condition RB )
Arithmetic expressions are just parsed with grammar production
Predicate -> Expr ( expression only ), so for this case now a
special handling has been added
( with new introduced method CegoPredicate::getExprOnlyValue )
lfcbase:
- To support several keyboard layouts, changed some window control
sequences from Cntrl-Key to ESC key
- Added virtual method hasData to Pager class. The method must be
implemented to indicate, if an execution returns a result set. If
no result set is returned, just a message box is shown and edit
mode is left ). This seems to be useful for DML queries ( insert,
udate, delete, create, drop, ...)
- In Pager class, added buffer scrolling in editor part. Preparation
started for Pager command history management
- Added history feature to Pager class. Command history can be loaded
and saved with readHistory and writeHistory methods.
Navigation through command history in Edit mode with Cntrl-P and Cntrl-N
cego:
- Adaptions made for pager and pager history feature
lfcbase:
- more patches for the Pager class
Removed init_color with custom color for select cursor since the
custom color seems to be poor supported for curses. Fixes added
to Pager::writeFooter
- edit-feature in Pager (must discuss this with Bjoern 8-)
- ignore SIGINT in Pager
- change from unsigned long long to long long value in Datetime class
- pattern search feature for Pager class
cego:
- Small fix in CegoBufferPool.h
- CegoClient adapted to new Pager API
- Queries now can be interrupted with Cntrl-C also in pager mode
- adapt to Datetime long long values ( e.g. date2str from int to long long )
lfcbase:
- Rework of Pager class, which is a API change
cego:
- Made some adaptions for reworked lfcbase Pager class
rows with multiline entries are more readable
- Added check script check119.sql to verify grammar expansion
for stored procedure condition
- CegoAction::procArg1() and CegoAction::procArg2() to check
duplicate parameter names
- Added check118.sql to demonstrate explicit table locking for nested
delete operation within a cursor loop
- Fix added in CegoFactor::getAggregationList:
for case QUERY and case FETCH, the not-supported exceptions
have been removed, otherwise this leads a useless exception
in CegoAttrCond::asConjunctionList where aggregations are just checked
- Added grammer change in Cego.def to ensure operator precedence
for AND and OR condition operator.
AND has higher precedence if no enclosing braces are given.
The result for the following queries should be the same
select * from t1 where a = 42 or a = 1 and b = 'X' ;
select * from t1 where a = 1 and b = 'X' or a = 42;
select * from t1 where a = 42 or ( a = 1 and b = 'X' );
- With cego version before 2.51.7, braces have been mandatory to
ensure operator precedence.
With the new grammar extension, the AND operator has a higher presedence
- CegoFactor::evalFieldValue, QUERY case, after query
evaluation, the parentJoinBuf has to be reset, otherwise an invalid
reference is used later to CegoSelect::getQueryId. This might result
in core dumps, if query cache is enabled
- Added querycache verification to db check routines
- Added querycache mode to command line options to set up query cache by batch
lfcbase:
- BigDecimal::div null value check
cego:
- CegoFactor::toChain to add braces for condition case
- CegoSelect::initAggregation, for sum and avg aggregation, aggregation
value is initialized with 0 and appropriate datatype ( instead of
null value ). This leads to numeric result 0 in case of empty result
sets ( instead of null value )
- Fix made in CegoDistCursor::distSetup to support FLA setting with
no cursor condition. This is needed for query plan evaluation for
nested inner/left outer/right outer joins
See check114 as a sample
devel/lfcbase 1.18.11 -> 1.19.0
- Changed Datetime::asLong to return signed long values ( long long ).
This is needed to support timestamp values before 01.01.1970,
which are signed negative
databases/cego 2.50.6 -> 2.51.0
- consolidate grammar productions for query conditions and
procedure conditions
- Since CegoPredicate::CompType has received new type EXPRONLY,
this requires a tableset export/import for upgrading to this version
( for predicate encoding / decoding )
- Several changed in CegoFieldValue to switch from Datetime integer
type unsigned long long to long long. This is required to support
datetime comparison for values before 01.01.1970
databases/cegobridge 1.8.1 -> 1.9.0
- adapt to cego 2.51.0
- Optimization added to CegoPredDesc::match. Matcher is just created,
if the match method is called, since it might occur, that the
condition is evaluated be CegoDistCursor via CegoAttrCond
- Fix in CegoAttrComp::getMatcher, there was still missing star
escape, which has been added to CegoPredDesc constructor. To avoid
two same pieces of code, CegoQueryHelper::sql2Regex has been
introduced
cego:
- Added syntax expansion for group having clause to support
advanced having conditions, e.g.
SELECT a, sum(b), max(c) FROM t1
GROUP BY a HAVING sum(b) > 10 AND max(c) = 100;
- Patch added for 'view condition evaluation' in
CegoAttrCond::asConjunctionList, CegoSelect::setViewCond and
CegoDistCursor::distSetup.
- Additional check added to CegoObjectManager to avoid duplicate
table objects of different type ( e.g. Table t1 and View t1 ). Table
objects are either native tables, views or aliases.
- For predicate export structure, changed from XML representation
to native SQL syntax. This requires a dedicated Parser in CegoXPorter,
which can handle predicate SQL representation, but saves lots of
code, since all subsequent member classes of CegoPredDesc no more
need to implement XML representation methods ( constructor,
fromElement, toElement )
- Added stabilization patches for nested grouping select, e.g. in
exists subselect. In CegoSelect::reset, for _selectMode == GROUPING,
grouping space is resetted now properly.
- Added stabilization patches to CegoAction. Load statements for
views, procedures, triggers and condition are treated regarding
object allocation/free. These statements are for internal use only,
so a database user receives a note message now
- Further improvements added in CegoQuery to give more detailed
error information in case of exceptions
- In CegoTableManager, made some exception message improvements,
instead of popping last message, the complete message stack is thrown
cegobridge:
- Support added for cego-2.50.x
- Fixes added for modified CegoAction::getSelect and
CegoAction::getProcedure methods
- For several modules, added exception trace information to improve
error logging in case of any exception
- Patch added for CegoQueryCache::addEntry, getNumUsed added to
search loop to find appropriate slot. This avoids some needless
assignments, since used entries cannot be assigned.
- Patch added CegoQueryCache::invalidate. For main loop, a delay
has been added, to give other threads the chance to claim the lock
- Patch for CegoFunction added.
For methods getReturnType, getReturnTypeLen and evelFieldValue
the following has been added : If proc object is getting used by
useObject, the following getProcedure call must be catched for
exception to unuse the object since _pProc is stiil null and the
object so is not unused by cleanup
- Removed lockPool call in CegoAdminThread::srvExportTableSet and
srvExportTable to avoid lock timeouts during long running exports
- Made some code cleanup in CegoAdminThread ( exception consolidation )
- db thread query history information, added timestamp and cost information
- Introduces new cgadm admin commands
o queryhist last n
o queryhist cost n
to show the query history for all db threads in the db thread
pool. The last n queries or the most expensive n queries are
indicated
- Added query history suppport for cgadm screen mode
- CegoAdmScreen to fully support of query history feature
( last and full over all db threads ).
- Screen2:Grid schema patches added to satisfy new column alignment
feature for Screen2
- Fix added in CegoDBThreadPool. CegoAdminThreadPool and CegoLogThreadPool.
If database hostname is not defined, the corresponding service is served
on any hostname. This avoids confusion regarding IPv4 and IPv6 host
configuration ( e.g. on MacOSX Sequoia, java seems to
handle localhost always as IPv4 127.0.0.1, while on POSIX layer,
it is mapped to ::1, a bit strange )
- Added new lfcbase Net::serve6 method to db, admin and log threadpool.
If no hostname is defined in database xml, an IP dual stack
configuration is set up.
- cgmkdb utility script:
allows to set up advanced database parameters
- In CegoXMLSpace::addTableSeDef, added ARCHMODE attribute to
set archmode to OFF as default
- Added querycache threshold parameter. Just queries with execution
time greater than threshold value are cached. This leads to a lower
stressed cache and avoids cache latency for those queries. Threshold
is given in msec, which seems to be appropriate for now.
- Some memory leaks fixed
- fix cgadm (hang if INTR received while idle)
- Added query history for db threads. Not just the last action is
indicated via admin command "threadinfo db threadId", but also the
last n queries processed by this thread. History Length can be
configured via new cego command line option dbthreadhist
- Patch added for memory leak in CegoSelect::cleanUp and
CegoSelect::reset, for both methods, _pCacheList has to checked and
freed, since for uncompleted queries ( e.g. exists clause ) the
_pCacheList was still not deleted
- Another fix for memory leak added in CegoFieldValue::getLocalCopy,
Before allocation, _len has to be checked for _len < STATICFIELDBUF
and then use _staticBuf, otherwise, the memory is lost
lfcbase:
- patch to Screen2::Form::handleKey to treat pipe character ( | )
- patch to Screen2::Attribute::setAttrList for refresh window
after attrlist has been updated
- Change in Screen2::Attribute class.
The class now provides three layout modes : HORIZONTAL, VERTICAL, OVERLAY.
In OVERLAY mode, with left and right cursor key it is switched
to visible attribute list. For this, the Screen2::Attribute constructor
has been changed
cego:
- For CegoAdmScreen, added QueryCacheInfo to tableset menu
- daptions made for CegoAdmScreen to
handle lfcbase-1.18.8 API change for Screen2::Attribute
lfcbase:
- Added patch in Matcher.cc to support extended regular expression
(added REG_EXTENDED flag to regcomp call )
cego:
- Added dedicated admin command to show query cache information
including hit rate ( show querycache for tableset )
This will replace the querycache information shown with general
show tableset admin
command
- Added patch for memory leak in CegoSelect. Within
CegoSelect::consolidateConjunctionList method, predicate and condition
instances are created for conjunction organisation. These instances
have to be registered and freed later within CegoSelect::cleanUp
- Rework of table cache logic, which is still experimental. A table
filter now can be defined to just cache a specified subset of tables.
Since table cache is just useful for specific tables with read only
flavor and medium size, this extension seems to makes sense. Cached
table rows can be accessed directly without accessing buffer pool
pages and without decoding. This might accelerate some database
configurations, where query cache is not appropriate and hot spot
readonly tables have to access very often and very fast.
- Added patches for querycache.
The CegoQueryCache::addEntry method now is limited to
QUERYCACHE_MAXTRY tries to add an entry. Otherwise numFail
counter is increaed, which is monitored for the query cache.
This should resolve deadlocks in smoother way ( no more lock timeouts )
in case of other db thread have the requested querycache slot in use.
For performance reasons, CegoQueryCacheEntry::calcSize was introduced
to just calculate the entry size in the constructor.
- Added lockId for lock objects in CegoBufferPool and CegoQueryCache
- Fix added for CegoXPorter / CegoAdminThread / CegoBufferPool
to treat locking issue during tableset / table import.
Since the pool is locked during import ( by new introduced method
lockPool ), no further locking calls must occur during import.
For this, log writing is disabled before lockPool is called and
CegoBufferPool::logIt checks for active log ( via new isActive
method ) before trying to get a lock.
- Synchronization of tableset admin operations added with this patch.
In CegoBufferPool, new methods are provided for tableset oriented locking
( lockPool and unlockPool ) These methods are used by CegoAdminThread
to synchronize concurrent tableset admin operations
( like stop tableset, drop tableset, export tableset, e.g. )
- Indent format improvements made for select, insert and update queries.
This is relevant for formatted sql code in stored procedures.
Author: Björn Lemke <lemke@lemke-it.com>
- More stabilization added to c wrapper library.
Relocation of cpluscheck and c wrapper check to root directory.
CegoCPlusCheck and cgwtest are now build during make all
Will be used in upcoming python module to access cego.
lfcxml:
- Patch added in XMLSuite.cc to treat CDATA lookahead correctly.
Trailing square brackets ( ] ) at the end of cdata caused parsing problems
cego:
- Added missing thread lock information for admin thread list and
thread stat command ( query + table locks, logmanager locks )
- Added patches in CegoQueryCache to improve locking.
For the addEntry method, the complete locking logic was
redesigned to avoid deadlocks
Author: Björn Lemke <lemke@lemke-it.com>
Date: Sat Jun 15 13:15:23 2024 +0200
- In CegoCaseCond, added getPlanList method to retrieve execution plan
for nested selects. The planlist then is catched in CegoFactor::getPlanList
- Consolidation done for data types decimal and fixed.
This version just supports the decimal datatype which originally was
the fixed datatype. The original decimal datatype was useless.
IMPORTANT: To migrate to version 2.49, you have to xml export
the databases and rename the datatype strings "fixed" and "FIXED"
to "decimal" and "DECIMAL"
- cegobridge needs to be rebuild
- Fix in CegoAdmScreen::TableSet::TableSet constructor,
SysUsage field added to indicate correct usage
for system, temp and app files
- Patch added for transaction handling in CegoTransactionManager::doCommit
and CegoTableManager::deleteDataTableEntry.
Just for tuple state COMMITTED, a rollback entry must be created,
otherwise double tuple delete may occur.
For tuple state INSERTED, tuple state is just set to OBSOLETE,
for tuple state OBSOLETE and DELETED, nothing has to be done.
In CegoTableManager::doCommit, tuple state just has to be set
to COMMITTED in case of INSERTED tuples
- Elimination of a cout in CegoQueryCache::QueryCacheEntry::cleanCache
- Added patches for return on update handling to treat value handling
in return list. The order in the return list must be stricter,
otherwise field evaluation via FLA caching in CegoFactor fails.
- returnList has been localized ( new method CegoQueryHelper::localizeFL )
to ensure correct values after update cursor ist closed.
- Patch added in CegoProcedure::cleanup to set pMasterBlock to zero.
In some cases, the setup master block lead to usage leaks for user
function calls. See check110.sql as a sample to cause this problem.
- Patch added in CegoAdminThread::srvParameterInfo to list current
QUEUEDELAY value and configured scandal formats ( SCANDATETIMEFORMAT )
- Small patch on CegoDistManager::startTableSet,
improvement for exception message added
Author: Bjoern Lemke <lemke@lemke-it.com>
- Patch added in CegoQueryHelper::evalBetween
The comparison ( f2 <= f1 && f1 <= f3 ) has changed to
( f1 >= f2 && f1 <= f3 )
This forces a type cast to f1 datatype in case mismatched datatypes.
This might be useful for queries like
select a from v1 where a between '30.11.2023' and '31.12.2023';
where a cast should be done to date time value ( instead of string )
Author: Björn Lemke <lemke@lemke-it.com>
- Patch added in CegoQueryHelper::encodeFL and CegoQueryHelper::decodeFL.
The dim field value was still not treated.
This might lead to wrong fixed values for update operations during
crash recovery operation
Author: Björn Lemke <lemke@lemke-it.com>
- Renaming date function dateformat to scandate to reflect its
scanning function. Please note, that cegojdbc uses this
function in CegoPreparedStatement class.
- Fix in CegoXPorter for view import.
For view creation, the field entries in schema must be setup
with a unique id ( analog to view creation via sql command )
Author: Björn Lemke <lemke@lemke-it.com>
- Add patch in CegoFactor::setFieldListArray, _flaCached again is
set to false since the field list in the the array may change
( e.g. for outer join queries used in views )
- In CegoDistCursor::getTuple, the tuple retrievel has been changed
for view objects. Complicated reasons...
lfcbase:
- Patch added in Sleeper class to support larger values for milliSleep
and microSleep. For this, also input parameter type has changed from
int to unsigned long
cego:
- Patch added for CegoCaseCond::evalField to eval appropriate datatype.
If the case condition contains null type this was not done correctly
- Added patch in CegoSelect::evalSchema. At the end of the method,
schema ids are calculated to provide unique attribute ids for views
- support nested case conditions
Author: Björn Lemke <lemke@lemke-it.com>
- Fix added in CegoProcQueryStmt::execute,
after execution of the query, the cleanUp method must be called
to avoid leaks regarding stored proc object use
( this corresponds to the new optimized stored proc handling )
- Patch addd for tableset stop procedure.
With the new optimized stored procedure handling ( CegoFunction module ),
the object cleanup must be done BEFORE the tableset is finally stopped.
Otherwise an uncaught exception occurs ( tableset not online )
which results in database crash after tableset shutdown.
Several modules have been modified for this
- In CegoFunction constructors, added _pProc = 0 initialization
for decoding and fromElement constructors. This avoids a set fault,
in case of database recovery, when a predicate expression
with user function is dedoded from log file
Author: Björn Lemke <lemke@lemke-it.com>
