An SQL Tuning Methodology

Note: This is just one of the various ways I capture and tune SQL. I use many different methods to capture suboptimal SQL depending on the urgency of the situation and the details I require to grab the SQL and work on a solution. It is not all encompassing. Db2 Design Advisor is your friend, and in my experience is not used enough!!! But the hard part comes when you have captured a suboptimal SQL and Design Advisor doesn’t have any recommendations!!!!  At this point you can’t just throw your hands up and give up, it has to be fixed!!!  This is where we shine with our clients, there isn’t an SQL out there that we can’t improve using all of the knowledge we have and all the tools in our toolbox. I’ll post more on SQL rewrite possibilities, index design and redesign, how to capture suboptimal SQL, statistics effects on optimization, statistical views, informational constraints, and hints, to name a few….

1. Connect to database and run “db2 call monreport.pkgcache” to get top statements.

 

— Command options —

CACHE_INTERVAL:                    All statements

SECTION_TYPE:                      Dynamic and static SQL

MEMBER:                            All

 

================================================================================

Part 1 – Summaries by ‘top’ metrics

 

Top 10 statements by TOTAL_CPU_TIME

——————————————————————————–

#   TOTAL_       STMT_TEXT

CPU_TIME

—  ———–  —————————————————————

1   217980426    INSERT INTO SPECTRUM_RESULT_5GHZ (WHEN,MEAS_ID_FK,EYE_ID,ANT_ID

2   137210647    select MEAS_ID from final table (INSERT INTO CLIENT_PCAP_RADIO

3   33451937     INSERT INTO SCAN_RADIO (NW_ID, AP_ID, EYE_ID, ANT_ID, CHANNEL,

4   24715177     INSERT INTO SPECTRUM_RESULT (WHEN,MEAS_ID_FK,EYE_ID,ANT_ID,SUBC

5   24691176     select MEAS_ID from final table (INSERT INTO RADIO_ENVIRONMENT_

6   18695817     INSERT INTO AP_FTP_QOS_TEST (WHEN,MEAS_ID,NW_ID,AP_ID,EYE_ID,AN

7   17015041     select MEAS_ID from final table (INSERT INTO AP_PCAP_RADIO (WHE

8   16529630     select MEAS_CATEGORY_ID from final table (INSERT INTO AP_FTP_QO

9   8262363      select MEAS_ID from final table (INSERT INTO CLIENT_PCAP_QOS (W

10  4794274      select MEAS_CATEGORY_ID from final table (INSERT INTO MOS_QOS_T

 

2. Go to topsql.sql script to get full SQL for above insert:

 

2017-09-22-13.09.08.775452             14586120             14586087

0                 5465                14507                    1

1                    2                    2                    2 INS

ERT INTO SCAN_RADIO (NW_ID, AP_ID, EYE_ID, ANT_ID, CHANNEL, HT_MODE, BEACON_OK,

SIGNAL_LEVEL_APEYE, SNR_APEYE, QUALITY, MAXQUAL, DURATION, WHEN ) VALUES ( ?, ?,

?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ? )

3. Then format the SQL for explain design advisor usage:

 

INSERT INTO CARAT7.SCAN_RADIO (NW_ID, AP_ID, EYE_ID, ANT_ID, CHANNEL,

HT_MODE, BEACON_OK,

SIGNAL_LEVEL_APEYE, SNR_APEYE, QUALITY,

MAXQUAL, DURATION, WHEN )

VALUES ( ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ? );

 

 

4. Run db2advis [db7sign@east1 ~]$ db2advis -d meas7 -i iscanradio.txt | more

 

Using user id as default schema name. Use -n option to specify schema

execution started at timestamp 2017-09-22-12.35.55.976835

found [1] SQL statements from the input file

Recommending indexes…

0  indexes in current solution

[ 11.0000] timerons  (without recommendations)

[ 11.0000] timerons  (with current solution)

[0.00%] improvement

 

 

—

—

— LIST OF RECOMMENDED INDEXES

— ===========================

—  no indexes are recommended for this workload.

 

 

—

—

— RECOMMENDED EXISTING INDEXES

— ============================

 

 

—

—

— UNUSED EXISTING INDEXES

— ============================

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_WHEN”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_APEYEWHENBEACON”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_APBEACONWHENNWSIG”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_EYENWBEACONWHENSIG”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_APNWBEACONWHENSIG”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_NWBEACONWHENSIG”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_APBEACONWHENNWSNR”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_APNWBEACONWHENSNR”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_NWBEACONWHENSNR”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_EYENWBEACONWHENSNR”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_APBEACONWHENNWCH”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_NWBEACONWHENCH”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_EYENWBEACONWHENCH”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_APNWWHENBEACON”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_EYENWWHENBEACONAP”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_WHENEYESIGHTCHANTAP”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_APEYEANTBEACONWHENSNR”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_EYEANTBEACONWHENSNR”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_EYECHBEACONWHENSNR”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_APWHENNWBEACON”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_CH_ANTID_APID_DBI3”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_EID_ANTID_APID_CH_DBI3”;

— DROP INDEX “CARAT7  “.”I_SCAN_RADIO_EYEWHEN”;

— ===========================

—

 

0 record(s) selected.

5. Review output and determine if index recommended or if not possible SQL rewrite or further manual analysis.
6. Review unused indexes on the table in question.

[db7sign@east1 ~]$ db2 “select lastused, indname from syscat.indexes where tabname=’SCAN_RADIO'” | more

 

LASTUSED   INDNAME

 

———- ———————————————————————

———————————————————–

09/21/2017 I_SCAN_RADIO_WHEN

 

09/22/2017 I_SCAN_RADIO_APEYEWHENBEACON

 

04/20/2017 I_SCAN_RADIO_APBEACONWHENNWSIG

 

04/17/2017 I_SCAN_RADIO_EYENWBEACONWHENSIG

 

09/22/2017 I_SCAN_RADIO_APNWBEACONWHENSIG

 

03/10/2017 I_SCAN_RADIO_NWBEACONWHENSIG

 

04/20/2017 I_SCAN_RADIO_APBEACONWHENNWSNR

 

11/04/2016 I_SCAN_RADIO_APNWBEACONWHENSNR

 

09/01/2017 I_SCAN_RADIO_NWBEACONWHENSNR

 

03/25/2017 I_SCAN_RADIO_EYENWBEACONWHENSNR

 

09/15/2017 I_SCAN_RADIO_APBEACONWHENNWCH

 

09/22/2017 I_SCAN_RADIO_NWBEACONWHENCH

 

09/08/2017 I_SCAN_RADIO_EYENWBEACONWHENCH

 

09/21/2017 I_SCAN_RADIO_APNWWHENBEACON

 

09/22/2017 I_SCAN_RADIO_EYENWWHENBEACONAP

 

09/07/2017 I_SCAN_RADIO_WHENEYESIGHTCHANTAP

 

03/31/2017 I_SCAN_RADIO_APEYEANTBEACONWHENSNR

 

09/14/2017 I_SCAN_RADIO_EYEANTBEACONWHENSNR

 

03/31/2017 I_SCAN_RADIO_EYECHBEACONWHENSNR

 

09/21/2017 I_SCAN_RADIO_APWHENNWBEACON

 

09/22/2017 I_SCAN_RADIO_CH_ANTID_APID_DBI3

 

09/22/2017 I_SCAN_RADIO_EID_ANTID_APID_CH_DBI3

 

09/22/2017 I_SCAN_RADIO_EYEWHEN

 

 

23 record(s) selected.

 

7. Review index definitions and compare to query.

Index                                                                                                                            Index                                                                                                                            Unique         Number of      Index                       Index          Null   Index           Data                                                                                                                                     Max Varchar     Xml                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  BUSINESS_TIME        Column

schema                                                                                                                           name                                                                                                                             rule           columns        type                        partitioning   keys   ID              type                                                                                                                             Hashed  Length          pattern                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              WITHOUT OVERLAPS     names

——————————————————————————————————————————– ——————————————————————————————————————————– ————– ————– ————————— ————– —— ————— ——————————————————————————————————————————– ——- ————— ——————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————– ——————– ——————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————–

CARAT7                                                                                                                           I_SCAN_RADIO_WHEN                                                                                                                D                           1 RELATIONAL DATA             –              Y                    1 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +WHEN

CARAT7                                                                                                                           I_SCAN_RADIO_APEYEWHENBEACON                                                                                                     D                           4 RELATIONAL DATA             –              Y                    2 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +AP_ID+EYE_ID+WHEN+BEACON_OK

CARAT7                                                                                                                           I_SCAN_RADIO_APBEACONWHENNWSIG                                                                                                   D                           5 RELATIONAL DATA             –              Y                    3 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +AP_ID+BEACON_OK+WHEN+NW_ID+SIGNAL_LEVEL_APEYE

CARAT7                                                                                                                           I_SCAN_RADIO_EYENWBEACONWHENSIG                                                                                                  D                           5 RELATIONAL DATA             –              Y                    4 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +EYE_ID+NW_ID+BEACON_OK+WHEN+SIGNAL_LEVEL_APEYE

CARAT7                                                                                                                           I_SCAN_RADIO_APNWBEACONWHENSIG                                                                                                   D                           5 RELATIONAL DATA             –              Y                    5 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +AP_ID+NW_ID+BEACON_OK+WHEN+SIGNAL_LEVEL_APEYE

CARAT7                                                                                                                           I_SCAN_RADIO_NWBEACONWHENSIG                                                                                                     D                           4 RELATIONAL DATA             –              Y                    6 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +NW_ID+BEACON_OK+WHEN+SIGNAL_LEVEL_APEYE

CARAT7                                                                                                                           I_SCAN_RADIO_APBEACONWHENNWSNR                                                                                                   D                           5 RELATIONAL DATA             –              Y                    7 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +AP_ID+BEACON_OK+WHEN+NW_ID+SNR_APEYE

CARAT7                                                                                                                           I_SCAN_RADIO_APNWBEACONWHENSNR                                                                                                   D                           5 RELATIONAL DATA             –              Y                    8 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +AP_ID+NW_ID+BEACON_OK+WHEN+SNR_APEYE

CARAT7                                                                                                                           I_SCAN_RADIO_NWBEACONWHENSNR                                                                                                     D                           4 RELATIONAL DATA             –              Y                    9 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +NW_ID+BEACON_OK+WHEN+SNR_APEYE

CARAT7                                                                                                                           I_SCAN_RADIO_EYENWBEACONWHENSNR                                                                                                  D                           5 RELATIONAL DATA             –              Y                   10 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +EYE_ID+NW_ID+BEACON_OK+WHEN+SNR_APEYE

CARAT7                                                                                                                           I_SCAN_RADIO_APBEACONWHENNWCH                                                                                                    D                           5 RELATIONAL DATA             –              Y                   11 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +AP_ID+BEACON_OK+WHEN+NW_ID+CHANNEL

CARAT7                                                                                                                           I_SCAN_RADIO_NWBEACONWHENCH                                                                                                      D                           4 RELATIONAL DATA             –              Y                   12 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +NW_ID+BEACON_OK+WHEN+CHANNEL

CARAT7                                                                                                                           I_SCAN_RADIO_EYENWBEACONWHENCH                                                                                                   D                           5 RELATIONAL DATA             –              Y                   13 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +EYE_ID+NW_ID+BEACON_OK+WHEN+CHANNEL

CARAT7                                                                                                                           I_SCAN_RADIO_APNWWHENBEACON                                                                                                      D                           4 RELATIONAL DATA             –              Y                   14 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +AP_ID+NW_ID+WHEN+BEACON_OK

CARAT7                                                                                                                           I_SCAN_RADIO_EYENWWHENBEACONAP                                                                                                   D                           5 RELATIONAL DATA             –              Y                   15 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +EYE_ID+NW_ID+WHEN+BEACON_OK+AP_ID

CARAT7                                                                                                                           I_SCAN_RADIO_WHENEYESIGHTCHANTAP                                                                                                 D                           7 RELATIONAL DATA             –              Y                   16 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +WHEN+EYE_ID+SIGNAL_LEVEL_APEYE+HT_MODE+CHANNEL+ANT_ID+AP_ID

CARAT7                                                                                                                           I_SCAN_RADIO_APEYEANTBEACONWHENSNR                                                                                               D                           6 RELATIONAL DATA             –              Y                   17 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +AP_ID+EYE_ID+ANT_ID+BEACON_OK+WHEN+SNR_APEYE

CARAT7                                                                                                                           I_SCAN_RADIO_EYEANTBEACONWHENSNR                                                                                                 D                           5 RELATIONAL DATA             –              Y                   18 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +EYE_ID+ANT_ID+BEACON_OK+WHEN+SNR_APEYE

CARAT7                                                                                                                           I_SCAN_RADIO_EYECHBEACONWHENSNR                                                                                                  D                           5 RELATIONAL DATA             –              Y                   19 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +EYE_ID+CHANNEL+BEACON_OK+WHEN+SNR_APEYE

CARAT7                                                                                                                           I_SCAN_RADIO_APWHENNWBEACON                                                                                                      D                           4 RELATIONAL DATA             –              Y                   20 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +AP_ID+WHEN+NW_ID+BEACON_OK

CARAT7                                                                                                                           I_SCAN_RADIO_CH_ANTID_APID_DBI3                                                                                                  D                           4 RELATIONAL DATA             –              Y                   21 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +CHANNEL+ANT_ID+WHEN+AP_ID

CARAT7                                                                                                                           I_SCAN_RADIO_EID_ANTID_APID_CH_DBI3                                                                                              D                           4 RELATIONAL DATA             –              Y                   22 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +EYE_ID+ANT_ID+CHANNEL+AP_ID

CARAT7                                                                                                                           I_SCAN_RADIO_EYEWHEN                                                                                                             D                           2 RELATIONAL DATA             –              Y                   23 –                                                                                                                                –                     – –                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    NO                   +EYE_ID-WHEN

 

8. Get additional index usage data from tcbstats.

 

Index usage analysis from tcbstats:

 

TCB Index Stats:

Address            TableName          IID   PartID EmpPgDel   RootSplits BndrySplts PseuEmptPg EmPgMkdUsd Scans      IxOnlyScns KeyUpdates InclUpdats NonBndSpts

PgAllocs   Merges     PseuDels   DelClean   IntNodSpl

 

0x00007F99D34C6EE0 SCAN_RADIO         25    n/a    0          0          9          0          0          550        328        0          0          134928     136332     0          0          0

1395

0x00007F99D34C6EE0 SCAN_RADIO         24    n/a    0          0          0          0          0          66         66         0          0          133191     133986     0          0          0

795

0x00007F99D34C6EE0 SCAN_RADIO         23    n/a    0          0          39         0          0          215        204        0          0          243578     247262     0          0          0

3645

0x00007F99D34C6EE0 SCAN_RADIO         22    n/a    0          0          31         0          0          120        120        0          0          241152     245526     0          0          0

4343

0x00007F99D34C6EE0 SCAN_RADIO         21    n/a    0          0          102        0          0          16         0          0          0          216808     220810     0          0          0

3900

0x00007F99D34C6EE0 SCAN_RADIO         20    n/a    0          1          68         0          0          0          0          0          0          518080     526974     0          0          0

8825

0x00007F99D34C6EE0 SCAN_RADIO         19    n/a    0          1          13962      0          0          310        310        0          0          427545     447655     0          0          0

6147

0x00007F99D34C6EE0 SCAN_RADIO         18    n/a    0          0          173        0          0          2          2          0          0          219850     223930     0          0          0

3907

0x00007F99D34C6EE0 SCAN_RADIO         17    n/a    0          0          39         0          0          6          6          0          0          243637     247319     0          0          0

3643

0x00007F99D34C6EE0 SCAN_RADIO         16    n/a    0          0          193        0          0          0          0          0          0          136991     139493     0          0          0

2309

0x00007F99D34C6EE0 SCAN_RADIO         15    n/a    0          0          364        0          0          128        128        0          0          134664     137137     0          0          0

2109

0x00007F99D34C6EE0 SCAN_RADIO         14    n/a    0          0          52         0          0          90         90         0          0          257142     261392     0          0          0

4198

0x00007F99D34C6EE0 SCAN_RADIO         13    n/a    0          0          214        0          0          118        118        0          0          179502     182811     0          0          0

3095

0x00007F99D34C6EE0 SCAN_RADIO         12    n/a    0          0          527        0          0          0          0          0          0          169237     172449     0          0          0

2685

0x00007F99D34C6EE0 SCAN_RADIO         11    n/a    0          1          54         0          0          0          0          0          0          438870     446363     0          0          0

7438

0x00007F99D34C6EE0 SCAN_RADIO         10    n/a    0          1          54         0          0          0          0          0          0          438871     446405     0          0          0

7479

0x00007F99D34C6EE0 SCAN_RADIO         9     n/a    0          0          507        0          0          0          0          0          0          167238     170397     0          0          0

2652

0x00007F99D34C6EE0 SCAN_RADIO         8     n/a    0          1          52         0          0          1          1          0          0          432856     440264     0          0          0

7355

0x00007F99D34C6EE0 SCAN_RADIO         7     n/a    0          0          191        0          0          4          4          0          0          175999     179221     0          0          0

3031

0x00007F99D34C6EE0 SCAN_RADIO         6     n/a    0          1          50         0          0          0          0          0          0          432808     440203     0          0          0

7344

0x00007F99D34C6EE0 SCAN_RADIO         5     n/a    0          0          38         0          0          1441       1398       0          0          244424     248014     0          0          0

3552

0x00007F99D34C6EE0 SCAN_RADIO         4     n/a    0          0          533        0          0          25         14         0          0          326689     332578     0          0          0

5356

0x00007F99D34C6EE0 SCAN_RADIO         3     n/a    0          0          1          0          0          2737       2737       0          0          133980     134785     0          0          0

804

0x00007F99D34C6EE0 SCAN_RADIO         2     n/a    0          1          292        0          0          25         25         0          0          399343     406545     0          0          0

6909

0x00007F99D34C6EE0 SCAN_RADIO         1     n/a    0          0          66704      0          0          517        500        0          0          26401      93604      0          0          0

499

0x00007F99D3AD61E0 SCAN_MANAGED       25    n/a    0          0          28         0          0          1431       1431       0          0          38589      39298      0          0

• 681

 

9. Develop recommended action based on all above analysis and recommend indexes to be dropped to application developers. Get feedback from developers on indexes in question.

 

10. Drop indexes and monitor post drop to ensure no negative impacts.

 

GOTO THE NEXT STATEMENT  and REPEAT UNTIL Tuning Complete or return not beneficial.