„Are your space management and monitoring tools up-to-date, or are they still DB2 V7?”
IFCIDs : Instant monitoring of secondary allocations
DB2 V7 : Space management without „the IBM algorithm“
DB2 V8 : Running out of datasets & Degenerated extents
DB2 9 : UTS spaces introduction & PBG’s problem
DB2 10 : ALTER PBGs
DB2 11 : Remove any empty parts in PBGs
SAX : Space management processing
Since DB2 V8, the DBAs of the world have all had the ability to forget about PQTY and SQTY in the DDL for Tablespaces and Indexes.
At first, nearly no-one trusted the sliding scale algorithm, and SOFTWARE ENGINEERING’s product Space AssuranceExpert (aka SAX) monitored and reacted instantly to secondary allocations.
However, we now have DB2 11 and I thought it would be interesting to review what was done in DB2 V7 (when our SAX was launched), and the difference nowadays in the DB2 11 world.
IFCIDs: Instant monitoring of secondary allocations
In DB2, every time a secondary allocation is done, DB2 can be made to spit out an IFCID. SAX runs as a started task, active 24×7, from DB2 start up until just before DB2 shut down.
It catches all of these IFCIDs thrown by DB2 and performs an analysis with five basic questions:
- Can this dataset reach its maximum physical size *before* running out of physical extents? (The actual size is dependent on the “geometry” of the object of course!)
- Will this object run out of datasets? (The number of datasets an object can have is, once again, dependent on the “geometry” of the object)
- Is this partition nearing its maximum size?
- Did DB2 ask for one extent but got more back?
- Are any of my SMS disk storage pools running out of space?
DB2 V7 : Space management without „the IBM algorithm“
Remember, back in those old days of DB2 V7, we only had 255 extents and 254 partitions, but datasets could still get pretty big pretty fast.
The problem lots of shops had, was that an important dataset would “hit the buffers” of maximum number of extents *way* before it ever ran out of physical space. Thus causing grief, wailing and gnashing of teeth!
SAX stopped all this by giving WTO “heads-up” style messages in two flavors.
– First a warning message,
– and then a critical message.
This gave DBAs and space managers much needed time to plan for the outage and the, inevitably, long running REORG to actually action the required ALTER, or perhaps even any DROP/RECREATE that had to be done.
DB2 V8 : Running out of datasets & Degenerated extents
IBM also noticed this problem and so introduced in DB2 V8 the “sliding scale” of secondary allocations, as long as the ZPARM OPTIMIZE EXTENT SIZING field (MGEXTSZ) was set to YES (this is the default from DB2 9, by the way).
Of course, to really use this, you then had to ALTER all of the existing spaces PQTY and SQTY to be -1, and then remember to delete all PRIQTY and SECQTY lines in your DDL and also rely on the TSQTY and IXQTY ZPARMs giving a big enough “first default”. (By the way, defaults for these two ZPARMS are 0, which is actually translated to be 720k or one cylinder for normal spaces and 7200k or 10 cylinders for LOB spaces). This all probably explains why the take up of this great feature has not been that spectacular and, in fact, Listserv *still* gets questions about “How good is this feature?”
This also explains why the primary reason for having SAX is still valid at most shops today!
Running out of datasets
Now the problem of running out of datasets is very, very evil indeed… For a non-partitioned space, you can have up to 32 datasets, and DB2 will happily allocate away and you will never know, or even be informed, if, and when, the last possible dataset has just been allocated and, of course, you will not know that the 33rd one cannot be allocated until you get a -904 unavailable resource error!
By definition this is “not good”, as you must do a HUGE REORG with a bunch of managers breathing down your neck and *not* make any mistakes with the new allocations. (Again, this is a very good reason to have SAX doing all the monitoring and triggering early warning “heads-up” style messages!)
A partition running out of space is rare, but when it does happen it is, of course, a disaster! The idea in SAX, is to warn when the end is near for any of the partitions in use and thus, as before, allow time for the ALTER etc.
Degenerated extents
Degenerated extents are annoying as well. You have 255 extents, DB2 requests one but gets up to five back! This is “wasting” your precious supply of extents and so SAX can also warn you if this starts happening, and remedial action can again be planned to correct the problem, (normally a volume defrag in this case).
Now in z/OS 1.7 “Extent Constraint Removal” was introduced for the DATACLAS which, if set to “Y”, allows 7,257 extents but still limits you to 123 extents per volume and 59 volumes, so watch out if you are using huge “virtual” disks (E.g. MOD 54 or EAV), as you can end up wasting space as you still cannot exceed 123 extents per volume.
Finally, when an SMS Pool runs out of space, either for sort/work or image copy, it is *not* good! The idea here, is to also give a “heads-up” style alert so that the DBA can trigger the space management people to have a look at the state and size of the SMS storage groups this time alerted by percentage used or GBs of space free.
DB2 V8 introduced a big change – Partitions went up to a maximum of 4,096, and the calculation about how many pieces your NPI can have got “a little bit complex” (see also my previous newsletter: “Are you going to PIECEs” here!).
DB2 9: UTS spaces introduction & PBG’s problem
In DB2 9 the next major advance came with UTS spaces, and the one that caused the most grief was, of course, PBG. Why?
Well, the first problem was that some people went mad and used MAXPARTITIONS 4096 right from the get-go which they then found out could not simply be changed and ended up being a huge problem.
IBM came out with a bunch of fixes for these people, but the recommendation is still true today: “Use the number you expect to use!”
PBGs, however, came with a new set of space management Problems:
- By definition every partition is full, and so a TP REORG is “dangerous” -especially if you have VARCHAR, and even more so if compressed.
- ALTER at TP level is not supported for PBG.
- Getting rid of empty partitions was not supported.
- Adding partitions dynamically (by command) was not supported.
- What to do if the partition that is “in use” is growing and is
- The last allowed partition
- MAXPARTITIONS is set to one?
Now these are “non trivial” because the DB2 catalog is so defined and you would not want an alert every time someone created a table or index!
The trick here, is to treat these conditions as if it was a normal space and so, instead of warning that you are using the last part, it waits until you are using e.g. 80% of that part and then e.g. at 90% comes the critical threshold warning.
DB2 10: ALTER PBGs
With DB2 10 came the ability to ALTER PBGs to add parts which made using DSN1COPY to clone data around a lot better!
DB2 11: Remove any empty parts in PBGs
Further, in DB2 11, the REORG utility can be used to remove any empty parts in PBGs by the use of the ZPARM REORG_DROP_PBG_PARTS being set to ENABLE (DISABLE is the default).
Space Management processing
The SAX tool way of processing all this info is neatly summarised in the help panel of the tool itself:
AUDIT SMS STOGROUPS
Should the Space AssuranceExpert audit SMS stogroups. Y/N
If Y is entered, a pop-up window will allow you to enter up
to 24 SMS storage groups which will be audited.
If WARN IF % ALLOC > or WARN IF GB FREE < is specified and
exceeded, a warning (WTO) will be issued.
AUDIT DEGENERATED XTS
Audit secondary quantity for de-generated extents. If
specified, a warning is issued in case of the last extent
does not reach this percentage of the SECQTY specified
in the DB2 catalog. If this field is left blank, no
auditing is performed.
SUPERVISE LPS
Supervise linear pagesets. If specified, a warning is issued
in case of high allocated reaches this percentage of the
maximum data set size for partitioned objects.
For non-partitioned objects, a warning is issued for every
newly allocated data set as soon as the data set number
reaches this percentage of the maximum number of data sets:
Two different values may be entered for warning and critical
values with different message ids. This may be useful for
automation reasons (see below).
Object type: TABLESPACE ! Maximum number of data sets
-----------------------------+----------------------------
LOB tablespaces ! 254
-----------------------------+----------------------------
Non-partitioned tablespaces ! 32
-----------------------------+----------------------------
Partitioned tablespaces ! 1 (Percent used check)
-----------------------------+----------------------------
Partitioned By Growth ! MAXPARTITIONS. LPS check if
tablespaces ! more than one. If on last
! partition then percent used.
-----------------------------+----------------------------
Object type: INDEX ! Maximum number of data sets
-----------------------------+----------------------------
Non-partitioned indexes on ! MIN ( 4096 , 2 power 32 /
tablespace with LARGE, ! ( DSSIZE / TS PGSIZE))
DSSIZE, or more than 64 ! Eg: 128 GB DSSIZE with
Partitions ! 8 KB Tablespace Page
! gives 256 Pieces (datasets)
! Or 4 GB DSSIZE with
! 4 KB Tablespace Page
! gives 4096 Pieces (datasets)
-----------------------------+----------------------------
Non-partitioned indexes ! 32
otherwise !
-----------------------------+----------------------------
Partitioned indexes ! 1 (Percent used check)
-----------------------------+----------------------------
To support automation based on WTO ids two different
thresholds may be specified:
Field (1) specifies a warning threshold using WTO ids
O2RTSU04 - 12W (non-partitioned spaces)
O2RTSU04 - 14W (partitioned spaces)
O2RTSU04 - 16W (partition by growth spaces)Field (2) specifies a critical threshold using WTO ids O2RTSU04 - 13W (non-partitioned spaces) O2RTSU04 - 15W (partitioned spaces) O2RTSU04 - 17W (partition by growth spaces)
So now you know why size matters for DB2!
The big question now is:
“Are your space management and monitoring tools up-to-date, or are they still DB2 V7?”
As usual any questions or comments are welcome,
TTFN Roy Boxwell
Senior Software Architect