1. Since ages Oracle has wallets that can be used to store userid and password configurations for connections in safe way. There are a few rules you need to follow but once you get them, they are pretty easy to use for any sqlnet client. Since zbxora is a good Oracle citizen it can use wallets without any problems.

    When a wallet is configured it is possible to connect in a way that is very similar to when OS authentication is used. When using OS authentication it is common to set ORACLE_SID and PATH and just issue a command like sqplus / to connect to your default database. In this case the default database is specified by the ORACLE_SID.

    For the configuration of Oracle wallet there are a few things to think about, like to which database should the connection go to and which database user should be used. They are both needed in the wallet configuration. It is important to know how the connection will be specified because the credentials are specified by the database url that is used. If for example in the wallet the database connect url is specified as //dbserver/service_name you had better make sure that the user also uses exactly this connection url, otherwise the wallet credentials are not used.

    configuration

    The configuration consists of 2 parts. One is the sqlnet.ora file, the other one is the wallet itself.

    sqlnet.ora

    The sqlnet.ora is found in the directory that is pointed to by the TNS_ADMIN environment variable. If this variable is not set then Oracle falls back to $ORACLE_HOME/network/admin, if the ORACLE_HOME environment variable is set. Other locations where sqlnet files are searched are /var/opt/oracle and /etc, depending on the operating system. Smartest is to define the TNS_ADMIN variable so it is clear for everyone where the slant configuration files are stored.

    In the sqlnet.ora file enter the wallet_location and the sqlnet.wallet_override parameters.

    WALLET_LOCATION = (SOURCE=(METHOD=FILE)(METHOD_DATA=(DIRECTORY=$HOME/etc)))
    SQLNET.WALLET_OVERRIDE = TRUE

    This configuration gives the option for every operating system user to have an own wallet located in their own etc directory.

    wallet

    The wallet contains connection data so it is not weird to have this file protected for modifications. If you want to edit this file, using the mkstore command, a password is required. The file itself can be copied to other locations and even other users without a problem. An admin can prepare the wallet files and distribute that to the users that need the connections, without those users having to know their own passwords.

    First the wallet needs to be created. If the wallet has to fit the sqlnet.ora as specified above, start creating an etc directory in the home directory. Next issue 

    mkstore -wrl etc -create
    

    and remember the password. This password is needed for each and every modification that needs to be done on this file using mkstore. mkstore is also used to add credentials.

    add credentials example:

    mkstore -wrl etc -createCredential //host.example.net/orcl cistats password_for_cistats

    to add the credentials to be used for a connection to //host.example.net/orcl. In this case the user that will be used is cistats and the password that will be used is password_for_cistats.

    In this case the database connect url is specified as //host.example.net/orcl but it could also have been specified as a tnsnames alias, in which case the tnsnames.ora is also needed to contain the connect description for that alias.

    to test the working, it is easiest to use a tool like sqlplus. 

    sqlplus /@//host.example.net/orcl

    should give a connection to the database. show user should - in this case - show CISTATS as connected user.

    The sqlplus connection syntax used is pretty simple: {username}/{password}@{db_url} When using a wallet, the username and the password are both empty and are found in the wallet using the db_url as the index.

    zbxora and Oracle wallet

    zbxora uses a connection configuration file that contains an entry for username and for password. Forcing zbxora to use the wallet is very simple, just leave the username and the password blank. What zbxora does is simply construct a string like {username}/{password}@{db_url} to connect, With both username and password being empty this results in /@{db_url} which triggers the sqlnet library to lookup the credentials to be used for the connection to {db_url}.

    maintaining the wallet

    mkstore does have a few extra commands to list and edit credentials. Useful are -listCredential that shows the db_urls and their configured username and -modifyCredential that can be used to change the credentials for a given db_url.

    0

    Een opmerking toevoegen

  2. I am using zabbix for about a year now and it still has not shown me all power that is inside. One of the things that is very interesting to know more about is the caching and the processing of log items. When something draws my attention that often this is because it has some problems.

    Log items are items that receive text lines from agents. Agents can be limited to send only a certain amount of log lines per second to try to prevent to overload the system. If for some reason there is a log storm where many logfiles get many new lines, they are sent to the zabbix_server and that stores them initially in the history cache, more precise into the History Write Cache, for which exists an internal item zabbix[wcache,values,log] that counts the log lines received.

    What you see during such a log storm is that the free space in the History Write Cache is quickly evaporating. When this cache and out of space, the server is starting to miss values, gaps are falling into the data, exactly what you don’t want to happen. Since the number of dbsyncers is a unable, it does not need a lot of thinking to experiment with that number. For this blog I tried with 2 and with 4 syncers and I did not find a real increase for the processing speed of log items.

    The only solution I found to prevent this slow down is to increase the cache size to such values that it takes hours to fill it up and hope that the log storms silences before the cache runs out of space.

    One of the items that seems to be missing is a counter for the data that is written to the database. Maybe it does exist, I am not aware of it. To  compensate for that I used zbxora to count the number of rows in the history_log table and present that as a speed to get number os rows per second. There is a process inserting in that table, there also is a process deleting from that table so the number of rows per second can also be negative. That means that more data is cleaned out than is inserted and after a log storm eventually that will happen.

    What happened during such a storm is shown in the following graphs.Cbr parc 001 08

    Here is shown the blue line that prints the percentage free of the Zabbix history write cache. It nose dived at 20160530 18:30 at that moment the cache was only 256M. At 21:30 I restarted the zabbix server with a cache of 512M. this gave a little more time but on 03:30 I restarted again, with the current setting, a cache size of 1024M.

    Also note that during the beginning of the log storm there also was a dip in the Value cache, which is logical because after a value has been added to the write cache, it also goes to the value cache, where it is cleaned after a few minutes.

    What happened in processing is visible here:

    Cbr parc 001 04

    What can be seen is a sudden rise in the values processed. This rise is caused by the number of Zabbix log values processed, the thin blue line … that is more or less stable at about 200 lines per second. The brown line shows the growth of the number of rows in the history_log table, counted using plain old SQL on the database. After a little peak of near 150 rows per second it slowed down to about 50 rows per second steady. This is the reason that the cache if getting full. For some reason the writing to the database is not very fast at all.

    The internal processing playing a role here is shown in the following  graph:Cbr parc 001 05

    The Zabbix busy history synced process, the red line is showing the process that does the writing to the database. Initially it was quiet. When the storm started, the synced went to 100% busy, for a very short while and slowed down to a steady 50%. until I changed the number of syncer processes from 2 to 4 at around 03:30, at the same time where I increased the cache value to 1G. From that point the activity went down to 25%. So, the synced process activity shows that only 1 process was busy with the syncing, most of the time.Both machines involved have plenty of CPU and memory idle. A simple imp test shows that importing a copy of the history_log tables using the same network runs with about 550 lines/s.

    It is also very visible that the syncers keep more or less busy processing the cached log values for hours after the log storm calmed down. No other items lost data because of the log items coming in. The reason that only 200 log lines per second came in is explained by the agent setting that allows it to send only a max of 100 lines per second. The storm happened on 2 machines concurrently.

    Questions that I have now are

    1. why is only 1 process active, when 4 are configured?
    2. what is the percentage of what that is shown in the Zabbix busy history synced processes?

    The database is an Oracle database and can handle a lots more inserts per socond. The inserts are done row by row, by issuing bulk inserts the efficiency can be improved but still, the insert rate is lower than I expect. I can see that the cache handling improved a lot since zabbix-2.4 but the syncers still need to get a little smarter to allow for a bigger throughput.

    My conclusion is that even tough the server made a big jump in handling log lines, the syncers need to become more active. They are doing some work but have a lot of room to do more work.

    1

    Opmerkingen tonen

  3. After a little adventure with a failed upgrade after which we needed to go back in time and open with reset logs, suddenly the zabbix monitoring picked up failing archive log backups. The logging showed:

     

    RMAN-00571: ===========================================================
    RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
    RMAN-00571: ===========================================================
    RMAN-03002: failure of backup command at 01/19/2016 13:17:44
    RMAN-06059: expected archived log not found, loss of archived log compromises recoverability
    ORA-19625: error identifying file +RECO/gabw1p/archivelog/2016_01_14/thread_1_seq_57729.32091.901135449
    ORA-17503: ksfdopn:2 Failed to open file +RECO/gabw1p/archivelog/2016_01_14/thread_1_seq_57729.32091.901135449
    ORA-15012: ASM file '+RECO/gabw1p/archivelog/2016_01_14/thread_1_seq_57729.32091.901135449' does not exist

     

    Behind the primary database there is a physical standby database with active data guard, open with realtime apply. The standby database was fully up to date so there was not a real problem but failing backups are not supposed to happen so what is happening, why and can we solve that? Lets start with a crosscheck:

    RMAN> crosscheck archive log all;

    RMAN-00571: ===========================================================
    RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
    RMAN-00571: ===========================================================
    RMAN-03009: failure of crosscheck command on ORA_DISK_8 channel at 01/19/2016 13:40:45
    ORA-19633: control file record 230341 is out of sync with recovery catalog

     

    What is in v$archived_log ?

    select * from v$archived_log where recid = 230341;

    RECID STAMP
    ---------- ----------
    NAME
    --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    DEST_ID THREAD# SEQUENCE# RESETLOGS_CHANGE# RESETLOGS RESETLOGS_ID FIRST_CHANGE# FIRST_TIM NEXT_CHANGE# NEXT_TIME BLOCKS BLOCK_SIZE CREATOR REGISTR STA ARC APPLIED DEL S COMPLETIO DIC DIC
    ---------- ---------- ---------- ----------------- --------- ------------ ------------- --------- ------------ --------- ---------- ---------- ------- ------- --- --- --------- --- - --------- --- ---
    END BACKUP_COUNT ARCHIVAL_THREAD# ACTIVATION# IS_ COM FAL END_OF_RED BAC
    --- ------------ ---------------- ----------- --- --- --- ---------- ---
    230341 901124839
    +RECO/GABW1P/ARCHIVELOG/2016_01_14/thread_1_seq_57661.972.901124839
    1 1 57661 1 30-JAN-14 838221116 1.4981E+10 14-JAN-16 1.4981E+10 14-JAN-16 555 512 ARCH ARCH NO YES NO NO A 14-JAN-16 NO NO
    NO 2 1 2373467580 YES NO NO NO

     

    Can we delete this record?  

     

    RMAN> change archivelog sequence 57661 incarnation 1 delete;

    Key Thrd Seq S Low Time
    ------- ---- ------- - ---------
    230341 1 57661 A 14-JAN-16
    Name: +RECO/GABW1P/ARCHIVELOG/2016_01_14/thread_1_seq_57661.972.901124839


    Do you really want to delete the above objects (enter YES or NO)? yes

    RMAN-06207: WARNING: 1 objects could not be deleted for DISK channel(s) due
    RMAN-06208: to mismatched status. Use CROSSCHECK command to fix status
    RMAN-06210: List of Mismatched objects
    RMAN-06211: ==========================
    RMAN-06212: Object Type Filename/Handle
    RMAN-06213: --------------- ---------------------------------------------------
    RMAN-06214: Archivelog +RECO/GABW1P/ARCHIVELOG/2016_01_14/thread_1_seq_57661.972.901124839

     

     I started with a crosscheck but make it a bit more specific now:

    RMAN> crosscheck archivelog from sequence 57661 incarnation 1 like '+RECO/GABW1P/ARCHIVELOG/2016_01_14/thread_1_seq_57661.972.901124839';

    RMAN-00571: ===========================================================
    RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
    RMAN-00571: ===========================================================
    RMAN-03009: failure of crosscheck command on ORA_DISK_8 channel at 01/19/2016 13:40:45
    ORA-19633: control file record 230341 is out of sync with recovery catalog

     

    Now try to mark is as unavailable:  

     

    RMAN> change archivelog sequence 57661 incarnation 1 unavailable;

    RMAN-00571: ===========================================================
    RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
    RMAN-00571: ===========================================================
    RMAN-03009: failure of unavailable command on ORA_DISK_1 channel at 01/19/2016 13:52:13
    ORA-19633: control file record 230341 is out of sync with recovery catalog

    RMAN> change archivelog sequence 57661 incarnation 1 uncatalog;

    RMAN-00571: ===========================================================
    RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
    RMAN-00571: ===========================================================
    RMAN-03009: failure of uncatalog command on ORA_DISK_1 channel at 01/19/2016 13:55:50
    ORA-19633: control file record 230341 is out of sync with recovery catalog

     

      

    weird. So validate control file will show an error?

    RMAN> validate current controlfile;

    channel ORA_DISK_1: starting validation of datafile
    channel ORA_DISK_1: specifying datafile(s) for validation
    including current control file for validation
    channel ORA_DISK_1: validation complete, elapsed time: 00:00:01
    List of Control File and SPFILE
    ===============================
    File Type Status Blocks Failing Blocks Examined
    ------------ ------ -------------- ---------------
    Control File OK 0 9770
    Finished validate at 19-JAN-16

     

      Oh and the error message looks like we are using an man catalog, which is nog the case, this is all running NOCATALOG.

     No. The control file appears to be valid although there is a corrupt record in it that is not manageable. Now let’s excersize patience and see if the database deletes this record based on the CONTROL_FILE_RECORD_KEEP_TIME parameter. The good news is: yes, in this case the database is able to auto fix the problem.

    conclusion

    Although the reason for this record to appear is unclear - probably a bug - it is nice to see that in the end it solves itself. In between make sure that the backups are completed and good for use. Oracle support had no solution for this. We could have re-created the control files but since the standby was happily running we decided to wait for the auto healing capabilities of the magical Oracle database.

    0

    Een opmerking toevoegen

  4. There are only a few metrics in zbxORA. So if you want to track something more specific, you might want to add some more. How to do this? No easier way than to give an example.

    Let’s track the usage of the Flash Recovery Area:

    select space_limit, space_used, space_reclaimable, number_of_files
    from v$recovery_file_dest

    this simple query will do. In zabbix terms, this query returns 4 items, 1 for every column. Those items have to be stored somewhere in zabbix and for that zabbix uses keys and hosts. A zabbix host can be anything and in this case, an Oracle database will be a zabbix host. So we can add the items to a host we already have in zabbix. This however takes a way a very powerful feature of zabbix: templates. Chances are that you want to track the FRA usage for multiple databases and having to add those items to each and every host/database what you want to track is not my idea of having a good time.

    receiving side

    template

    So, instead, create a template and create your items in a template and assign the template to your hosts that you want to track. Templates are created in the Configuration/Templates menuCbr parc 001 03

    At the right hand side you find the Create Template button.Cbr parc 001 04

     

    Use that to create a brand new template to contain the latest and greatest items to track.Cbr parc 001 05 The Add button is found on the bottom of the page.Cbr parc 001 06

    This gives us an nice clean template:Cbr parc 001 07

    It has no Applications, Items, Triggers, it is completely empty, as expected. It is also not - yet - linked to any host.

    create items

    Select the Items link to get into the items create area.Cbr parc 001 08

    Cbr parc 001 09

    Again, on the right side is the Create button, this time for the Items.

    Now comes the hard part, thinking about names.Cbr parc 001 10

    name

    Give the item a name and a key. They can be the same and personally, I keep them the same, this saves having to think about new names. The key is important, since the data will be uploaded to zabbix using a host, key and value set for every item. The host is not known at this moment, that comes after having added the template to a host.

    Type

    Also important is the Type, default ‘Zabbix Agent’ but in our case this needs to be changed to ‘Zabbix Trapper’. Zabbix Trapper items are items that can be uploaded using zabbix_sender, a tool that zbxORA does use for this purpose.

    Type of Information

    Most of the times this will be a choice of Numeric (unsigned) or Numeric (Float). Use what best fits the data, Numeric(unsigned) when possible.

    Data Type

    in our case this will be Decimal.

    Units

    Since the usage is specified in Bytes, this will be ‘B’ If your query happen to return MB instead of simple bytes, you still use ‘B’ and use the ‘Use Custom Multiplier’ to fix that.

    Use Custom Multiplier

    if you did return MB, enter 1048567 (being the product of 1024 * 1024), if you returned Bytes, just leave this empty. When the data is in, zabbix uses this to make a human readable display of the amounts. GB worth of Bytes will be shown as GB and not as a huge amount of Bytes.

    The remainder of the fields are nice to play with enough is written about their usage.

    query results

    The query returns 1 row with 4 columns. This needs to be changed to 4 rows of 2 columns, first column will be the item key and the second column will be the value. Here is the relation between items and the query.

    one way to rewrite this query can be:

    select 'fra[limit]', space_limit from v$recovery_file_dest def
    union all
    select 'fra[used]', space_used from v$recovery_file_dest def
    union all
    select 'fra[reclaimable]', space_reclaimable from v$recovery_file_dest def
    union all
    select 'fra[files]', number_of_files from v$recovery_file_dest def

    I have chosen the items keys ‘fra[limit]’, ‘fra[used]’, ‘fra[reclaimable]’ and ‘fra[files]’. The keys can be anything, as long as they are the same in zabbix as in the query result. Spaces are allowed but in that case the query also has to generate the surrounding quotes because otherwise zabbix_sender won’t see the key as a single column.

    in my case the output is:

    fra[limit]        5368709120000
    fra[used]         83856719872
    fra[reclaimable]  78523662336
    fra[files]        326

    Enter the 4 items Cbr parc 001 11

    In the end the items show:Cbr parc 001 12

    Don’t forget: the used and reclaimable columns are percentages and need a float as data type. Limit and files can do with integers.

    Now the template is ready, assign it to the hosts, the databases that you want to monitor. In this example I use testhost.Cbr parc 001 13

    There are more ways to assign a template to a host …. The receiving side is ready now. Time to find a way to send the data, from the sending side.

    sending side

    On the sending side, you already have zbxORA running. If not, here http://ronr.blogspot.nl/2015/09/install-python-without-root-access-to.html is a description of how to get zbxORA running.

    zbxORA uses a simple file structure and a simple configuration for the connections. In the connection part of the configuration you define the username, password and connect url for the database but also the name of site specific checks files that you might want to use.

    The connection part is in etc/zbxORE.{db_name}.cfg

    and the checks are located in etc/zbxora_checks/oracle/

    $>ls etc/zbxora_checks/oracle/
    asm.11.cfg      ebs.cfg         primary.11.cfg  sap.cfg         standby.11.cfg
    asm.12.cfg      ebs.cfg.example primary.12.cfg  sap.cfg.example standby.12.cfg

    Here is were the default checks are and here is where you can add your ‘my_super_checks.cfg'

    Of course, if you find that your checks should be included in the project, feel free to push them back using git, in that case make sure you prepare them for inclusion into the files where they belong, according to their version and the database role where you want to monitor.

    Now add them to my_super_checks.cfg:

    [super_checks]
    minutes: 05
    my_qry1: select 'fra[limit]', space_limit from v$recovery_file_dest def
             union all
             select 'fra[used]', space_used from v$recovery_file_dest def
             union all
             select 'fra[reclaimable]', space_reclaimable from v$recovery_file_dest def
             union all
             select 'fra[files]', number_of_files from v$recovery_file_dest def

    The format is very similar to an old init style of format and don’t forget to forget the terminating ‘;’ it is not needed. Also keep the indentation as shown, the parsing of the file will fail if you SQL continues on the start of a next line.

    There is a requirement for a section name and every section should have a minutes parameter where is defined how big the monitoring interval should be. In this case that is every 5 minutes but it could also have been 1 minute, or 13, or 123 minutes, just what you want.

    There is one special case of minutes: the 0 minutes case

    startup section

    If you want some queries to only run when zbxORA successfully connected to the database, give it an minutes parameter with value 0. This causes the checks in the mentioned section to be run only once every connection.

    special section names

    section names almost have no meaning. There is one exception, that is sections with ‘discover’ in their name. All other sections have simple data returning queries in them, the discover sections have the magical zabbix LLD queries in them. LLD is short for Low Level Discovery. In the discovery you pass to zabbix the dynamic contents of databases where you want to generate items for. The default set of checks are full of them. As you can imagine, the resulting data is treated slightly different and is passed as json arrays to zabbix, using same mechanism as the data. The key that is mentioned in the LLD part of the template is the same the name for the discovery queries in the checks files. For the discovery this is a requirement.

    re-reading config

    Now your checks are in place. zbxORA still does not know about them. The process is already running and checking the files that were known at connection time for changes. If it finds a file changed, zbxORA wil re-read the files and start using the changes right away. Now you added a new file so you need to make this fact known to zbxORA. For that we have the site_checks parameter in the connection parameter file. There just add:

    site_checks: my_super_checks

    mostly the base name part of your file.

    There are 2 ways to make zbxORA to re-read the complete config. One is to kill and restart the process, the other is to kill the currently running Oracle session that zbxORA uses. In both cases it will re-read the config in order to find the needed parameters to build a new connection.

    zabbix_sender

    Make sure you have zabbix_sender installed on your system and that zbxORA has it in the PATH of the runtime environment.

    The easiest way to send the data to zabbix is using the following config:

    hostname: testhost
    checks_dir: etc/zbxora_checks
    out_dir: $HOME/zbxora_out
    site_checks: my_super_checks
    to_zabbix_method: zabbix_sender
    to_zabbix_args: zabbix_sender -z 127.0.0.1 -T -i

    The easiest way and when there is just one database. This causes a new zabbix_sender process every time data is to be sent to zabbix. This already is in batches but one batch for every database, one zabbix_sender for every database. If you have a lot of databases, more efficient is to disable zabbix_sender from zbxORA and use zbxora_sender to combine the data for all databases. This example assumes that your zbxORA process[es] are running on the zabbix server or on a zabbix proxy. This might be different in your case but having them on a proxy might be a good location. If it is different, change the -z parameter to whatever address the proxy or server is running on. The parameter to_zabbix_args is not parsed by zbxORA, it is just passed as is to zabbix_sender.

    For now, I think this is enough to get one started. If you hit any problems, feel free to contact. Downloading and using zbxORA is free, clone it from github

    0

    Een opmerking toevoegen

  5. I recently uploaded zbxORA, a simple Oracle monitoring plugin for zabbix to github

    The plugin is written in python and uses cx_Oracle for database connectivity so it makes sense to explain how to install a recent version of python on a machine. If you happen to have root access, it is quite simple, if not, read along, I will show how to install locally without root access. Assumption made is that the machine already contains a c compiler.

    For this case I am going to install python into $HOME/ciber, where the python executable ends up in $HOME/ciber/bin

    python 2.7.10

    download the sources from

    • https://www.python.org/ftp/python/2.7.10/Python-2.7.10.tgz
    • http://mirror.proserve.nl/debian/pool/main/z/zlib/zlib_1.2.8.dfsg.orig.tar.gz
    • https://pypi.python.org/packages/source/s/setuptools/setuptools-18.1.tar.gz
    and place them in a temporary location, for example $HOME/tmp/
     
    The setup tools need the zlib library. The setup tools are not really needed to run zbxora but if you want to add python packages, they might come in handy so start with compiling zlib. This saves a runtime error when trying to use the setup tools.
     
    gzip -dc zlib_1.2.8.dfsg.orig.tar.gz|tar -xf -
    cd gzip-1.2.8
    ./configure —prefix=$HOME/ciber
    make
    make install
    cd

    Now unpack and compile python:

    gzip -dc Python-2.7.10.tgz|tar -xf -
    cd Python-2.7.10
    ./configure —prefix=$HOME/ciber
    make
    make install
    cd

    Simple, isn’t it? Without the configure this was a nightmare. There are a few linker errors, they are no problem when using python to run just zbxORA

    Now setup and install the setup tools:

    export PATH=$HOME/bin:$HOME/ciber/bin:$PATH
    unzip -qq setuptools-18.1.tar.gz|tar -xf -
    cd setuptools-18.1
    python setup.py build
    python setup.py build

    As said before, not needed for zbxora but you never know, if you learn to appreciate the power and readability of python, there will be more packages coming into you system soon ...

    What really is needed is the Oracle client software. Without this, no serious connection to Oracle databases is possible. Luckily for us, the instant client is free downloadable and usable. Start with downloading it from

    Oracle client
    • http://download.oracle.com/otn/linux/instantclient/121020/instantclient-basic-linux.x64-12.1.0.2.0.zip
    • http://download.oracle.com/otn/linux/instantclient/121020/instantclient-sdk-linux.x64-12.1.0.2.0.zip
    mkdir oracle
    cd oracle
    unzip -qq ../instan*.zip
    cd instantclient_12_1
    ln -sf libclntsh.so.12.1 libclntsh.so
    cd
    mv oracle $HOME/ciber/
    ORACLE_HOME=$HOME/ciber/oracle/instantclient_12_1

    We are going to use $HOME/ciber/oracle as ORACLE_HOME. ORACLE_HOME is quite an important variable in Oracle environments. From this point we find the binaries, libraries and error messages. In a clean setup the binaries are in $ORACLE_HOME/bin and the libraries in $ORACLE_HOME/lib. To keep this simple, we keep it al in just ORACLE_HOME. Make sure this variable is defined in your .bash_profile.

    Now download and compile cx_Oracle

    • https://pypi.python.org/packages/source/c/cx_Oracle/cx_Oracle-5.2.tar.gz
    gzip -dc cx_Oracle-5.2.tar.gz|tar -xf -
    cd cx_Oracle-5.2/
    python setup.py install

    Now we are ready to run a python Oracle client.

    zbxORA

    Download zbxora from 

    • https://github.com/ikzelf/zbxora

    unzip the archive and place the *.cfg files in $HOME/ciber/etc and the scripts in $HOME/ciber/bin. Make sure that $HOME/ciber/bin is in your PATH, otherwise even python would not be found.

    $>zbxora.py -c etc/zbxora.fsdb02.cfg
    2015-09-11 22:14:53 start zbxora-0.39 pid=27634 Connecting...
    2015-09-11 22:14:53.793066 site_checks: sap,ebs
    2015-09-11 22:14:53.793083 to_zabbix_method: NOzabbix_sender zabbix_sender -z 127.0.0.1 -T -i /Users/ronr/zbxora_out/zbxora.fsdb02.zbx
    2015-09-11 22:14:53.793096 out_file:/Users/ronr/zbxora_out/zbxora.fsdb02.zbx
    2015-09-11 22:14:54.818389: (1.1)connection error: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor for cistats@//localhost:15214/fsdb02
    2015-09-11 22:14:56.412204: (2.1)connection error: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor for cistats@//localhost:15214/fsdb02
    2015-09-11 22:14:57.986881: (3.1)connection error: ORA-12514: TNS:listener does not currently know of service requested in connect descriptor for cistats@//localhost:15214/fsdb02
    ^CTraceback (most recent call last):
      File "bin/zbxora.py", line 385, in
        time.sleep(SLEEPER)
    KeyboardInterrupt

    If everything is OK, your log results should be similar to those listed above. This simply means that everything is working but the database url specified in etc/zbxora.fsdb02.cfg is not correct. Make sure that the parameter is pointing to a database and you will see other messages like userid or password are wrong … ;-)

    I hope this little note does help you getting started with python for Oracle.

    0

    Een opmerking toevoegen

  6. I just had to do this again and had some time digging before it went smooth. In that case it is good to document the process for later use. This time I will be using the Oracle Instance Client 11.2.0.4.0 as can be found on OTN in http://www.oracle.com/technetwork/topics/intel-macsoft-096467.html Take good care for picking the correct word size, in most cases it will be the 64 bits variant that I will also use for this short article.

    NewImage

    You might as well download them all but needed for DBD::Oracle are

    1. instantclient-basic-macos.x64-11.2.0.4.0.zip
    2. instantclient-sdk-macos.x64-11.2.0.4.0.zip
    3. instantclient-precomp-macos.x64-11.2.0.4.0.zip
    4. instantclient-sqlplus-macos.x64-11.2.0.4.0.zip If you also want to be able to run sqlplus
    5. instantclient-tools-macos.x64-11.2.0.4.0.zip for an extra RAT client
    6. instantclient-jdbc-macos.x64-11.2.0.4.0.zip is needed if you need an XA compliment adbc driver

    In this case I downloaded them all and found them in the Downloads directory.

    To unpack and arrange them in to a usable location:

    cd Downloads
    ls -lrt instantclient-*
    -rw-r--r--@ 1 ronr staff 62794903 Oct 22 17:02 instantclient-basic-macos.x64-11.2.0.4.0.zip
    -rw-r--r--@ 1 ronr staff 884608 Oct 22 17:04 instantclient-sqlplus-macos.x64-11.2.0.4.0.zip
    -rw-r--r--@ 1 ronr staff 651903 Oct 22 17:05 instantclient-sdk-macos.x64-11.2.0.4.0.zip
    -rw-r--r--@ 1 ronr staff 235951 Oct 22 17:05 instantclient-tools-macos.x64-11.2.0.4.0.zip
    -rw-r--r--@ 1 ronr staff 7836666 Oct 22 17:06 instantclient-precomp-macos.x64-11.2.0.4.0.zip
    -rw-r--r--@ 1 ronr staff 1560298 Oct 23 21:28 instantclient-jdbc-macos.x64-11.2.0.4.0.zip
    ls instantclient-*.zip|xargs -n 1 unzip -qq

    This leaves us with a directory containing it all:

    ls -lrt instantclient_11_2/
    total 391264
    -r--r--r--@ 1 ronr staff 368 Apr 11 2011 glogin.sql
    -r--r--r--@ 1 ronr staff 10564 Dec 18 2012 cobsqlintf.o
    -r--r--r--@ 1 ronr staff 82856 Aug 24 2013 orai18n-mapping.jar
    -r--r--r--@ 1 ronr staff 1655734 Aug 24 2013 orai18n.jar
    -r-xr-xr-x@ 1 ronr staff 159004 Jan 7 2014 libocijdbc11.dylib
    -r-xr-xr-x@ 1 ronr staff 13744 Jan 7 2014 libheteroxa11.dylib
    -r--r--r--@ 1 ronr staff 66779 Jan 7 2014 xstreams.jar
    -r-xr-xr-x@ 1 ronr staff 1504252 Jan 7 2014 libsqlplusic.dylib
    -r-xr-xr-x@ 1 ronr staff 1365444 Jan 27 2014 libsqlplus.dylib
    -r-xr-xr-x@ 1 ronr staff 8744 Jan 27 2014 sqlplus
    -r--r--r--@ 1 ronr staff 2091135 Jan 28 2014 ojdbc5.jar
    -r--r--r--@ 1 ronr staff 2739616 Jan 28 2014 ojdbc6.jar
    -r-xr-xr-x@ 1 ronr staff 2817872 Jan 29 2014 libnnz11.dylib
    -r-xr-xr-x@ 1 ronr staff 66167420 Feb 7 2014 libclntsh.dylib.11.1
    -r-xr-xr-x@ 1 ronr staff 1897664 Feb 7 2014 libocci.dylib.11.1
    -rwxrwxrwx@ 1 ronr staff 40768 Apr 10 2014 genezi
    -rwxrwxrwx@ 1 ronr staff 118707148 Apr 10 2014 libociei.dylib
    -rwxrwxrwx@ 1 ronr staff 14348 Apr 10 2014 adrci
    -rwxrwxrwx@ 1 ronr staff 754728 Apr 10 2014 wrc
    -rwxrwxrwx@ 1 ronr staff 162380 Apr 10 2014 uidrvci
    -rw-rw-rw-@ 1 ronr staff 484 Apr 10 2014 BASIC_README
    -rw-rw-rw-@ 1 ronr staff 482 Apr 10 2014 JDBC_README
    -rw-rw-rw-@ 1 ronr staff 488 Apr 10 2014 SQLPLUS_README
    drwxrwxrwx@ 3 ronr staff 102 Apr 10 2014 precomp
    -rw-rw-rw-@ 1 ronr staff 484 Apr 10 2014 TOOLS_README
    -rw-rw-rw-@ 1 ronr staff 3121 Apr 10 2014 PRECOMP_README
    drwxrwxrwx@ 9 ronr staff 306 Oct 23 21:31 sdk

    I like to install my things orderly, like we are used to do with the rdbms software. That means I won’t install in /usr because that makes upgrading a nightmare since during upgrades /usr might be re-created from scratch, meaning that what you installed in there is gone. For me reason enough to install In /Applications.

    ORACLE_HOME=/Applications/oracle/product/instantclient_64/11.2.0.4.0
    cd instantclient_11_2/
    mkdir -p $ORACLE_HOME/bin
    mkdir -p $ORACLE_HOME/lib
    mkdir -p $ORACLE_HOME/jdbc/lib
    mkdir -p $ORACLE_HOME/rdbms/jlib
    mkdir -p $ORACLE_HOME/sqlplus/admin
    mkdir -p $ORACLE_HOME/sdk

    The landing zone’s are in place now, let’s put the files where they belong:

    mv ojdbc* $ORACLE_HOME/jdbc/lib/
    mv *.jar $ORACLE_HOME/rdbms/jlib/
    mv glogin.sql $ORACLE_HOME/sqlplus/admin/
    mv *dylib* $ORACLE_HOME/lib/
    mv *README $ORACLE_HOME/
    mv precomp $ORACLE_HOME/
    mkdir -p $ORACLE_HOME/precomp/lib
    mv cobsqlintf.o $ORACLE_HOME/precomp/lib/
    mv sdk $ORACLE_HOME/
    mv * $ORACLE_HOME/bin/

    Now the files are where they belong. Don’t be too quick, if you want to do nice stuff for which you need to compile libraries, the loader will refuse to load the libraries as they are now. Fix that.

    cd $ORACLE_HOME/lib
    ls -l
    total 376272
    -r-xr-xr-x@ 1 ronr staff 66167420 Feb 7 2014 libclntsh.dylib.11.1
    -r-xr-xr-x@ 1 ronr staff 13744 Jan 7 2014 libheteroxa11.dylib
    -r-xr-xr-x@ 1 ronr staff 2817872 Jan 29 2014 libnnz11.dylib
    -r-xr-xr-x@ 1 ronr staff 1897664 Feb 7 2014 libocci.dylib.11.1
    -rwxrwxrwx@ 1 ronr staff 118707148 Apr 10 2014 libociei.dylib
    -r-xr-xr-x@ 1 ronr staff 159004 Jan 7 2014 libocijdbc11.dylib
    -r-xr-xr-x@ 1 ronr staff 1365444 Jan 27 2014 libsqlplus.dylib
    -r-xr-xr-x@ 1 ronr staff 1504252 Jan 7 2014 libsqlplusic.dylib

    The most used libraries libclntsh and libocci won’t be recognized by the loader now. There is an easy fix for that:

    ln -sf libclntsh.dylib.11.1 libclntsh.dylib
    ln -sf libocci.dylib.11.1 libocci.dylib

    This leaves us with a directory that looks like this:

    ls -l
    total 376288
    lrwxr-xr-x 1 ronr wheel 20 Oct 23 22:01 libclntsh.dylib -> libclntsh.dylib.11.1
    -r-xr-xr-x@ 1 ronr staff 66167420 Feb 7 2014 libclntsh.dylib.11.1
    -r-xr-xr-x@ 1 ronr staff 13744 Jan 7 2014 libheteroxa11.dylib
    -r-xr-xr-x@ 1 ronr staff 2817872 Jan 29 2014 libnnz11.dylib
    lrwxr-xr-x 1 ronr wheel 18 Oct 23 22:02 libocci.dylib -> libocci.dylib.11.1
    -r-xr-xr-x@ 1 ronr staff 1897664 Feb 7 2014 libocci.dylib.11.1
    -rwxrwxrwx@ 1 ronr staff 118707148 Apr 10 2014 libociei.dylib
    -r-xr-xr-x@ 1 ronr staff 159004 Jan 7 2014 libocijdbc11.dylib
    -r-xr-xr-x@ 1 ronr staff 1365444 Jan 27 2014 libsqlplus.dylib
    -r-xr-xr-x@ 1 ronr staff 1504252 Jan 7 2014 libsqlplusic.dylib

    Now it should be possible to install DBD::Oracle. In DBD::Oracle-1.74 there still is a small problem, the Makefile.PL is looking in several places for the  makefile the is needed in Oracle related projects but not in $ORACLE_HOME/sdk/demo/demo.mk Until that is fixed, the installation of DBD::Oracle will fail in the link phase but again, there is an easy fix for that:

    First make the DBD::Oracle module as far as it goes using cpan, the perl package manager:

    cpan -m DBD::Oracle
    CPAN: Storable loaded ok (v2.22)
    Reading '/Users/ronr/.cpan/Metadata'
    Database was generated on Thu, 23 Oct 2014 14:17:02 GMT
    CPAN: Module::CoreList loaded ok (v2.50_02)
    Running make for module 'DBD::Oracle'
    CPAN: Digest::SHA loaded ok (v5.47)
    CPAN: Compress::Zlib loaded ok (v2.024)
    Checksum for /Users/ronr/.cpan/sources/authors/id/P/PY/PYTHIAN/DBD-Oracle-1.74.tar.gz ok
    CPAN: Archive::Tar loaded ok (v1.54)
    CPAN: File::Temp loaded ok (v0.22)
    CPAN: YAML loaded ok (v1.13)
    CPAN: Parse::CPAN::Meta loaded ok (v1.40)
    Configuring P/PY/PYTHIAN/DBD-Oracle-1.74.tar.gz with Makefile.PL
    Using DBI 1.631 (for perl 5.012005 on darwin-thread-multi-2level) installed in /opt/local/lib/perl5/site_perl/5.12.5/darwin-thread-multi-2level/auto/DBI/

    Configuring DBD::Oracle for perl 5.012005 on darwin (darwin-thread-multi-2level)

    Remember to actually *READ* the README file! Especially if you have any problems.

    Installing on a darwin, Ver#13.1 Using Oracle in /Applications/oracle/product/instantclient_64/11.2.0.4.0 DEFINE _SQLPLUS_RELEASE = "1102000400" (CHAR) Oracle version 11.2.0.4 (11.2)

    Unable to locate an oracle.mk or other suitable *.mk file in your Oracle installation. (I looked in /Applications/oracle/product/instantclient_64/11.2.0.4.0/rdbms/demo/demo_xe.mk /Applications/oracle/product/instantclient_64/11.2.0.4.0/rdbms/lib/oracle.mk /Applications/oracle/product/instantclient_64/11.2.0.4.0/rdbms/demo/oracle.mk /Applications/oracle/product/instantclient_64/11.2.0.4.0/rdbms/demo/demo_rdbms.mk /Applications/oracle/product/instantclient_64/11.2.0.4.0/rdbms/demo/demo_rdbms64.mk /Applications/oracle/product/instantclient_64/11.2.0.4.0/rdbms/lib/ins_rdbms.mk under /Applications/oracle/product/instantclient_64/11.2.0.4.0)

    The oracle.mk (or demo_rdbms.mk) file is part of the Oracle RDBMS product. You need to build DBD::Oracle on a system which has one of these Oracle components installed. (Other *.mk files such as the env_*.mk files will not work.) Alternatively you can use Oracle Instant Client.

    In the unlikely event that a suitable *.mk file is installed somewhere non-standard you can specify where it is using the -m option: perl Makefile.PL -m /path/to/your.mk

    See the appropriate README file for your OS for more information and some alternatives.

    at Makefile.PL line 1187. Warning: No success on command[/opt/local/bin/perl5.12 Makefile.PL] PYTHIAN/DBD-Oracle-1.74.tar.gz /opt/local/bin/perl5.12 Makefile.PL -- NOT OK

    That - as expected fails to find the demo.mk because it is not looking for it and certainly not in the right places. Now enter cpan and use the shell:

    cpan
    Terminal does not support AddHistory.

    cpan shell -- CPAN exploration and modules installation (v2.05) Enter 'h' for help.

    cpan[1]> look DBD::Oracle Reading '/Users/ronr/.cpan/Metadata' Database was generated on Thu, 23 Oct 2014 14:17:02 GMT Running look for module 'DBD::Oracle'

    Trying to open a subshell in the build directory... Checksum for /Users/ronr/.cpan/sources/authors/id/P/PY/PYTHIAN/DBD-Oracle-1.74.tar.gz ok Scanning cache /Users/ronr/.cpan/build for sizes ............................................................................DONE Working directory is /Users/ronr/.cpan/build/DBD-Oracle-1.74-OrEq9L

    vi Makefile.PL
    /mk_oci64

    NewImage

    and insert ‘sdk/demo/demo.mk’ in the list for my @mk_oci64 giving:

    NewImage

    save the file and still from the cpan shell issue

    perl Makefile.PL

    and now it - amongst others - shows:

    DEFINE _SQLPLUS_RELEASE = "1102000400" (CHAR)
    Oracle version 11.2.0.4 (11.2)
    Found /Applications/oracle/product/instantclient_64/11.2.0.4.0/sdk/demo/demo.mk
    Using /Applications/oracle/product/instantclient_64/11.2.0.4.0/sdk/demo/demo.mk

    Since we now have a makefile that has the correct location for the oracle makefile in it we can really build the perl module we need. Still from the cpan shell issue:

    make

    And this now generates an enormous amount of output and ends with:

    cp Oracle.bs blib/arch/auto/DBD/Oracle/Oracle.bs
    chmod 644 blib/arch/auto/DBD/Oracle/Oracle.bs
    Manifying blib/man3/DBD::Oracle::Troubleshooting.3pm
    Manifying blib/man3/DBD::Oracle::Troubleshooting::Cygwin.3pm
    Manifying blib/man3/DBD::Oracle.3pm
    Manifying blib/man3/DBD::Oracle::Troubleshooting::Sun.3pm
    Manifying blib/man3/DBD::Oracle::Troubleshooting::Vms.3pm
    Manifying blib/man3/DBD::Oracle::Troubleshooting::Hpux.3pm
    Manifying blib/man3/DBD::Oracle::Troubleshooting::Macos.3pm
    Manifying blib/man3/DBD::Oracle::Object.3pm
    Manifying blib/man3/DBD::Oracle::Troubleshooting::Aix.3pm
    Manifying blib/man3/DBD::Oracle::Troubleshooting::Linux.3pm
    Manifying blib/man3/DBD::Oracle::Troubleshooting::Win64.3pm
    Manifying blib/man3/DBD::Oracle::Troubleshooting::Win32.3pm
    Manifying blib/man3/DBD::Oracle::GetInfo.3pm
    sudo make install

    places the DBD::Oracle module in the correct location and

    exit

    takes us out of the cpan shell.

    I hope this article helps making Oracle project on the mac a breeze.

    update for El Capitan

    Since El Capitan it seems that DYLD_LIBRARY_PATH is no longer used to search for libraries to load. Could be a bug, could be on purpose. A way to get around this is to create symlinks for the libraries in /usr/local/lib/ or just copy the lib files:

    cp $ORACLE_HOME/lib/* /usr/local/lib/

    was enough to get cx_Oracle for zbxora.py functional again. Not exactly a nice solution if you want to be able to run various versions of the client next to each other for testing reasons.

    1

    Opmerkingen tonen

  7. why

    Up until recently I created a new vm using the graphical installer, every time selecting the same options, the same packages and every time manually. Now vagrant, puppet and ansible are getting more and more popular it was time to automate this a little. Since setting up a vagrant box also takes quite some time I opted for kickstart. In my case I will quicker install a new version than the time needed to create a new vagrant box than that I can re-use. So kickstart gives me quicker results.

    Before I had it all up and running, there flowed a lot of water through the river Rhine, and through many others too… So I will make a quick note from what I made and with a little luck will it safe a lot of time for others who have similar issues. For a die hard sysadmin this will mostly be a piece of cake. My confession here is that I am not a sysadmin, I am just a humble dba who needs quick and good vm’s for testing.

    My goal is to setup a vm with a minimal install, for which I don’t want to do anything manually until the OS is installed and ready to receive the Oracle installations. The way to go is to use kickstart and tftp. Virtualbox has a tftp server built in, that I am going to use. It does only work when using NAT networking. This alone took me quite a while to find ….

    The tools I use for this article are

    1. a macbook with
    2. macos 10.9.5
    3. VirtualBox 4.3.16 and
    4. Oracle Linux 7.0

    The last part, the v7.0 part is what caused me to dive into ansible to prepare the OS for it’s first Oracle installation. This because unlike OL6, for v7 there is not - yet - an Oracle pre install package.

    The steps

    1. setup the tftp server in VirtualBox
    2. create the kickstart script
    3. script the vbox creation

    setup the tftp server in VirtualBox

    To be able to setup an usable tftp server, you do need running Linux installation to pick some files from. The files you need come from the syslinux package so if it is not present yet, install that. This gives us the pxelinux0 file and the menu.c3 files that are needed.

    /usr/share/syslinux/pxelinux.0
    /usr/share/syslinux/menu.c32

    The other 2 files can be copied from an OL installer disc.

    /mnt/dvd/images/pxeboot/initrd.img
    /mnt/dvd/images/pxeboot/vmlinuz

    As you might have guessed, the installer image is mounted on /mnt/dvd/

    On the mac, the TFTP directory that VirtualBox creates to serve tftp from can be found in $HOME/Library/VirtualBox/ Just copy the pxelinux.0 and the menu.c32 files in the root of the TFTP directory.

    scp -p linuxhost:/usr/share/syslinux/pxelinux.0 ~/Library/VirtualBox/TFTP/
    scp -p linuxhost:/usr/share/syslinux/menu.c32 ~/Library/VirtualBox/TFTP/
    mkdir ~/Library/VirtualBox/TFTP/pxelinux.cfg
    mkdir -p ~/Library/VirtualBox/TFTP/images/OEL/x86_64/7.0/
    scp -p linuxhost:/mnt/dvd/images/pxeboot/initrd.img ~/Library/VirtualBox/TFTP/images/OEL/x86_64/7.0/
    scp -p linuxhost:/mnt/dvd/images/pxeboot/vmlinuz ~/Library/VirtualBox/TFTP/images/OEL/x86_64/7.0/

    This should end up with a structure similar to TFTP contents

    Now all that is left is the file that controls what happens when a vm is booted from the network, the TFTP/pxelinux.cfg/default file. IN my case, the content is:

    timeout 10
    default menu.c32

    menu title ==== Boot Menu ====
    label 1
      menu label ^ 1) OL65-Server install
        kernel images/OEL/x86_64/6.5/vmlinuz
          append initrd=images/OEL/x86_64/6.5/initrd.img inst.stage2=hd:LABEL=Oracle Linux 6.5\x20Server.x86_64 ks=http://thuis/ol65-anaconda-ks.cfg ramdisk_size=10000 repo --name="UEK3 kernel repo" --baseurl=cdrom://sr0:/mnt/source  dns=192.168.2.6 gateway=192.168.2.1 netmask=255.255.255.0 ksdevice=eth1

    label 2
      menu default
      menu label ^ 2) OL70-Server install
        kernel images/OEL/x86_64/7.0/vmlinuz
          append initrd=images/OEL/x86_64/7.0/initrd.img inst.stage2=hd:LABEL=OL-7.0\x20Server.x86_64 ks=http://thuis/ol70-anaconda-ks.cfg ramdisk_size=10000 repo=cdrom

    This presents a little menu that gives a little chance to select for an OL65-Server install or a OL70-Server install. The “menu default” directive specifies which choice is the default that will be picked when there was no reaction within the timeout period.

    If the default  is chosen, stage2 of the boot process will continu from the DVD location specified by “LABEL=OL-7.0 Server.x86_64” This is the contents of the .discinfo file found in the root of the installer image. Make sure all files are readable for others.

    create the kickstart script

    in the default menu, the kickstart script location is specified as ks=http://thuis/ol70-anaconda-ks.cfg. Easiest is to copy this from a previous installed images where you can find it in the home directory of root. I did add a few minimal things to get ansible up and running the quickest. Here is the content of my kickstart file:

    #version=RHEL7
    # System authorization information
    auth --enableshadow --passalgo=sha512
    poweroff

    # Use CDROM installation media
    cdrom
    # Run the Setup Agent on first boot
    firstboot --enable
    ignoredisk --only-use=sda
    # Keyboard layouts
    keyboard --vckeymap=us --xlayouts='us'
    # System language
    lang en_US.UTF-8

    # Network information
    network  --bootproto=dhcp --device=enp0s3 --onboot=on --ipv6=auto --activate --hostname=ol70.local
    network  --bootproto=dhcp --device=enp0s8 --onboot=on --ipv6=auto --activate --hostname=ol70.local
    # Root password
    rootpw --iscrypted $6$sWj3/0XiHWHmW2XZ$nK5FI1/qmW9S8pk.O6gA9bDlrdW7keLcgPKU0
    # System services
    services --enabled="chronyd"
    # System timezone
    timezone Europe/Amsterdam --isUtc
    user --name=ronr --password=$6$R7/Ec.JmIBHHE.fLlb5nayLZuFQjf0 --iscrypted --gecos="Ronald Rood"
    # System bootloader configuration
    bootloader --location=mbr --boot-drive=sda
    autopart --type=lvm
    # Partition clearing information
    clearpart --linux --initlabel
    %packages
    @core
    chrony
    wget

    %end
    %post --log=/root/my-post-log
    %include http://thuis/ol70-post-config
    %end

    In the post-config I need wget so it must be installed before it can be used. In the post-install you can do whatever you want, I did the minimal to make it possible to use ansible to configure the rest of the machine.

    The post-config:

    #
    # create ansible user, fill in authorized_keys and sudoers file
    #
    /usr/sbin/useradd -m --gid wheel --groups wheel --comment 'Anisble Configuration' --uid 1342 ansible
    /bin/mkdir /home/ansible/.ssh
    wget http://thuis/key_ronr.pub
    mv key_ronr.pub /home/ansible/.ssh/authorized_keys
    /bin/chown -R ansible /home/ansible/.ssh
    /bin/chmod -R 700 /home/ansible/.ssh
    echo "ansible  ALL=(ALL)       NOPASSWD: ALL" >>/etc/sudoers
    wget http://thuis/ol70_host_keys.tar.gz
    gzip -dc ol70_host_keys.tar.gz|(cd /etc;tar -xf -)

    It copies back the host keys, if I save them before, it stores the ssh keys for the ansible user and add’s the ansible user to the sudoers file. The reason to copy the eventually saved host keys is that I did make many runs before everything worked and I got sick of that ssh message telling me there is possibly a man in the middle. For a totally fresh new host, make it generate new keys, as is done by default.

    script the vbox creation

    Now that most setup is done, it is time to create a fresh new VM. This I also scripted and I think it is not a perfect script but it sure does do a lot of smart things. Again, since I had many runs before it all worked as I wanted, I did save a previous configuration and re-used the network mac addresses so I would get back the same IP-addresses from the internal dhcp server.

    My setup in general has 2 network cards, 1 for host only and 1 for bridged networking. In order to be able to use the PXE boot with VirtualBox, initially I create the first adapter to be used with NAT and later, when the VM is ready, I change it back to host-only. The script is placed in ~/bin/mkvm and has to be called with the name of the vm to be generated and the iso image to be used to install from. The script does not modify the magical default of the TFTP/pxelinux.cfg/default file so that must be setup manually, otherwise you will possibly end up with a wrong version of OS installed on the brand new VM.

    The script: clone it from git using

    git clone https://github.com/ikzelf/VirtualBox.git
    ISO_BASE=/Users/Shared/software/oracle/OEL
    VBOX_APPL_RES="/Applications/VirtualBox.app/Contents/MacOS/"
    VBOX_VM_BASE=${VBOX_VM_BASE-"$HOME/VirtualBox VMs"}
    VBOX_INFO=${VBOX_INFO-"$HOME/vbox"}
    KICK_START_KEYS=thuis:/Library/Server/Web/Data/Sites/Default/

    if [ $# -ne 2 ]
    then
    echo "$0 vmname install.iso" >&2
    echo " to be created" >&2
    echo " where install.iso is one of `ls ${ISO_BASE}`" >&2
    exit 1
    fi
    vmname=$1
    isoname=$2
    VM_INFO="$VBOX_INFO/${vmname}"

    if [ ! -d "$VBOX_VM_BASE" ]
    then
    echo "$0 directory $VBOX_VM_BASE nonexistent" >&2
    exit 1
    fi

    if [ ! -f ${ISO_BASE}/${isoname} ]
    then
    echo "$0 iso $isoname not in $ISO_BASE" >&2
    exit 2
    fi

    if [ -d "$VBOX_VM_BASE/${vmname}" ]
    then
    echo "$0 ${vmname} already in $VBOX_VM_BASE" >&2
    exit 3
    fi

    if [ -f "$VBOX_VM_BASE/${vmname}/${vmname}.vbox" ]
    then
    echo "$0 ${vmname} already in $VBOX_VM_BASE" >&2
    exit 3
    fi

    if [ ! -d "$VM_INFO" ]
    then
    mkdir -p "$VM_INFO"
    fi

    if [ -f "${VM_INFO}/${vmname}.info" ]
    then
    echo "$0 Re using settings from previous $vmname"
    if [ -f "${VM_INFO}/${vmname}_host_keys.tar.gz" ]
    then
    scp -p "${VM_INFO}/${vmname}_host_keys.tar.gz" "$KICK_START_KEYS"
    fi
    fi

    bridge_nic=`ifconfig | pcregrep -M -o '^[^\t:]+:([^\n]|\n\t)*status: active'| \
    grep "^[a-z].*:"|cut -f1 -d":"|sort|head -1`
    case $bridge_nic in
    en0) type=Ethernet;;
    en1) type="Wi-Fi (AirPort)";;
    *) type=geenID;;
    esac

    echo "$0 for Bridged using $bridge_nic $type"

    VBoxManage createvm --name "$vmname" --register
    if [ $? -ne 0 ]
    then
    echo "$0 createvm for ${vmname} failed" >&2
    exit 10
    fi

    VBoxManage modifyvm "$vmname" --memory 1024
    VBoxManage modifyvm "$vmname" --cpus 2
    VBoxManage modifyvm "$vmname" --ioapic on
    VBoxManage modifyvm "$vmname" --acpi on
    VBoxManage modifyvm "${vmname}" --boot1 disk --boot2 net --boot3 DVD --boot4 none
    #
    # # kickstart from tftp only supported using NAT
    #
    VBoxManage modifyvm "${vmname}" --nic1 NAT
    VBoxManage modifyvm "${vmname}" --nattftpfile1 /pxelinux.0
    VBoxManage modifyvm "${vmname}" --nic2 bridged --bridgeadapter2 "${bridge_nic}: ${type}"

    VBoxManage modifyvm "$vmname" --nictype1 "82540EM"
    VBoxManage modifyvm "$vmname" --nictype2 "82540EM"

    if [ -f "$VM_INFO/${vmname}.info" ]
    then
    echo "$0 Re using saved macadresses"
    grep "^NIC" $VM_INFO/${vmname}.info|grep " MAC:"|cut -f1 -d","|sed "s/:/ /g"|
    while read x NICn x MACa
    do
    echo "$0 Re using $MACa for NIC${NICn}"
    VBoxManage modifyvm "${vmname}" --macaddress${NICn} "${MACa}"
    done
    fi

    VBoxManage modifyvm "$vmname" --ostype "Oracle_64"
    #
    VBoxManage storagectl "${vmname}" --name "SATA Controller" --add sata
    VBoxManage storagectl "${vmname}" --name "IDE Controller" --add ide
    # # create disks
    VBoxManage createvdi --filename "$VBOX_VM_BASE/${vmname}/${vmname}-disk01.vdi" --size 64000 # 64g
    VBoxManage createvdi --filename "$VBOX_VM_BASE/${vmname}/${vmname}-disk02.vdi" --size 64000 # 64g
    VBoxManage createvdi --filename "$VBOX_VM_BASE/${vmname}/${vmname}-disk03.vdi" --size 64000 # 64g

    VBoxManage storageattach "${vmname}" --storagectl "SATA Controller" \
    --port 0 --device 0 --type hdd \
    --medium "$VBOX_VM_BASE/${vmname}/${vmname}-disk01.vdi"

    VBoxManage storageattach "${vmname}" --storagectl "SATA Controller" \
    --port 1 --device 0 --type hdd \
    --medium "$VBOX_VM_BASE/${vmname}/${vmname}-disk02.vdi"

    VBoxManage storageattach "${vmname}" --storagectl "SATA Controller" \
    --port 2 --device 0 --type hdd \
    --medium "$VBOX_VM_BASE/${vmname}/${vmname}-disk03.vdi"

    VBoxManage storageattach "${vmname}" --storagectl "IDE Controller" \
    --port 0 --device 1 --type dvddrive \
    --medium "${ISO_BASE}/${isoname}"
    ## --medium emptydrive
    ## /dev/sr0

    ## boot devices .... no cd no cdrom
    echo "$0 Installing ${vmname} using kickstart form TFTP"
    VBoxHeadless --startvm "${vmname}"
    echo not $0 VBoxManage startvm "${vmname}"
    ##
    ## the kickstart script will shutdown the guest
    ##
    echo "$0 Basic OS install of ${vmname} is complete replace NAT interface by hostonly"
    ## convert nic1 back to hostonly
    VBoxManage modifyvm "$vmname" --nic1 hostonly --hostonlyadapter1 vboxnet0
    VBoxManage showvminfo ${vmname} >$VM_INFO/${vmname}.info

    #
    # # replace installer disc by guest host additions disc
    #
    VBoxManage storageattach "${vmname}" --storagectl "IDE Controller" \
    --port 0 --device 1 --type dvddrive \
    --medium "$VBOX_APPL_RES/VBoxGuestAdditions.iso"
    ## --medium emptydrive
    ## device0 /dev/sr0
    ## device1 /dev/sr1
    VBoxManage startvm "${vmname}"
    #
    # # use ansible to finish the setup, including guest additions
    #

    In the script you will see there are 2 ways to start the VM. The first one is to start it headless. Doing so ensures that the script waits until the kickstart script is finished and shuts down the VM. After that, the network adapter is changed to host-only and the VBoxGuestAdditions.iso is inserted into the cdrom drive. If for any reason the scripts get’s into an interactive menu, you can always use rdp to connect to the VM and respond to the menus. The kickstart options are a bit sensitive so this might be a way to help the process.

    Another nifty thing that the script does is adjust the nattftpfile to /pxelinux.0. The default is something like /vmname.pxe and I did not like that.

    For me this blog also serves my memory, I hope this is also usable for others. The script can also be downloaded from github

    0

    Een opmerking toevoegen

  8. Every once and a while Oracle visit customers sites to check the customers license compliance. Often this means squashing extra cash form the customer, who is often not aware of the rules and how licensing works. This is not very strange, I don’t think anybody completely oversees Oracle licensing policy. In a pre check we found some unexpected and unused options installed. In earlier versions this  meant a deinstall and re-install of the software, this time a bit more careful.

    In 11g there is a nifty utility that makes this a bit easier: chopt. It is found in ORACLE_HOME/bin/. It chops options from the active ORACLE_HOME

    test112 ronr@sol11:~
    $>chopt

    usage:

    chopt <enable|disable> <option>

    options:
    dm = Oracle Data Mining RDBMS Files
    dv = Oracle Database Vault option
    lbac = Oracle Label Security
    olap = Oracle OLAP
    partitioning = Oracle Partitioning
    rat = Oracle Real Application Testing

    e.g. chopt enable rat

    The usage also show the options for which this tool can be used.

    In order to run this, the instances running from the ORACLE_HOME that has the unlicensed option activated, should al be taken offline. This is because in the end, the ORACLE_HOME/bin/oracle binary is re-built. If everything goes as it should be, you get to see the make output lines. To my big surprise, chopt was very quick on my Solaris on intel test system:

    test112 ronr@sol11:~
    $>chopt disable rat

    Writing to /home/ronr/app/ronr/product/11.2.0/dbhome_1/install/disable_rat.log...
    %s_unixOSDMakePath% -f /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/ins_rdbms.mk rat_off
    %s_unixOSDMakePath% -f /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/ins_rdbms.mk ioracle

    and nothing more. The lines are very familiar if you ever ran make manually to rebuild the oracle executable. For others this might not like something that makes a lot of sense but it does.

    check before disable:

    SQL> col parameter form a25
    SQL> col parameter value form a10
    SQL> col value form a10
    SQL> select * from v$option where parameter = 'Real Application Testing';

    PARAMETER VALUE
    ------------------------- ----------
    Real Application Testing TRUE
    test112 ronr@sol11:~
    $>chopt disable rat

    Writing to /home/ronr/app/ronr/product/11.2.0/dbhome_1/install/disable_rat.log...
    %s_unixOSDMakePath% -f /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/ins_rdbms.mk rat_off
    %s_unixOSDMakePath% -f /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/ins_rdbms.mk ioracle

    Normally I expect that chopt automagically runs the following make commands but in my case I had to run this manually.

    test112 ronr@sol11:~
    $>make -f /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/ins_rdbms.mk rat_off
    /usr/ccs/bin/ar d /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/libknlopt.a kecwr.o
    /usr/ccs/bin/ar cr /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/libknlopt.a /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/kecnr.o
    test112 ronr@sol11:~
    $>make -f /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/ins_rdbms.mk ioracle
    chmod 755 /home/ronr/app/ronr/product/11.2.0/dbhome_1/bin

    - Linking Oracle
    rm -f /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/oracle
    /usr/ccs/bin/ld -o /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/oracle -L/home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/ -L/home/ronr/app/ronr/product/11.2.0/dbhome_1/lib/ -dy /home/ronr/app/ronr/product/11.2.0/dbhome_1/lib/prod/lib/amd64/crti.o /home/ronr/app/ronr/product/11.2.0/dbhome_1/lib/prod/lib/amd64/crt1x.o /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/opimai.o /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/ssoraed.o /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/ttcsoi.o -z allextract -lperfsrv11 -z defaultextract /home/ronr/app/ronr/product/11.2.0/dbhome_1/lib/nautab.o /home/ronr/app/ronr/product/11.2.0/dbhome_1/lib/naeet.o /home/ronr/app/ronr/product/11.2.0/dbhome_1/lib/naect.o /home/ronr/app/ronr/product/11.2.0/dbhome_1/lib/naedhs.o /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/config.o -lserver11 -lodm11 -lcell11 -lnnet11 -lskgxp11 -lsnls11 -lnls11 -lcore11 -lsnls11 -lnls11 -lcore11 -lsnls11 -lnls11 -lxml11 -lcore11 -lunls11 -lsnls11 -lnls11 -lcore11 -lnls11 -lclient11 -lvsn11 -lcommon11 -lgeneric11 -lknlopt `if /usr/ccs/bin/ar tv /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/libknlopt.a | grep xsyeolap.o > /dev/null 2>&1 ; then echo "-loraolap11" ; fi` -lslax11 -lpls11 -lplp11 -lserver11 -lclient11 -lvsn11 -lcommon11 -lgeneric11 `if [ -f /home/ronr/app/ronr/product/11.2.0/dbhome_1/lib/libavserver11.a ] ; then echo "-lavserver11" ; else echo "-lavstub11"; fi` `if [ -f /home/ronr/app/ronr/product/11.2.0/dbhome_1/lib/libavclient11.a ] ; then echo "-lavclient11" ; fi` -lknlopt -lslax11 -lpls11 -lplp11 -ljavavm11 -lserver11 -lwwg `cat /home/ronr/app/ronr/product/11.2.0/dbhome_1/lib/ldflags` -lncrypt11 -lnsgr11 -lnzjs11 -ln11 -lnl11 -lnro11 `cat /home/ronr/app/ronr/product/11.2.0/dbhome_1/lib/ldflags` -lncrypt11 -lnsgr11 -lnzjs11 -ln11 -lnl11 -lnnz11 -lzt11 -lztkg11 -lmm -lsnls11 -lnls11 -lcore11 -lsnls11 -lnls11 -lcore11 -lsnls11 -lnls11 -lxml11 -lcore11 -lunls11 -lsnls11 -lnls11 -lcore11 -lnls11 -lztkg11 `cat /home/ronr/app/ronr/product/11.2.0/dbhome_1/lib/ldflags` -lncrypt11 -lnsgr11 -lnzjs11 -ln11 -lnl11 -lnro11 `cat /home/ronr/app/ronr/product/11.2.0/dbhome_1/lib/ldflags` -lncrypt11 -lnsgr11 -lnzjs11 -ln11 -lnl11 -lnnz11 -lzt11 -lztkg11 -lsnls11 -lnls11 -lcore11 -lsnls11 -lnls11 -lcore11 -lsnls11 -lnls11 -lxml11 -lcore11 -lunls11 -lsnls11 -lnls11 -lcore11 -lnls11 `if /usr/ccs/bin/ar tv /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/libknlopt.a | grep "kxmnsd.o" > /dev/null 2>&1 ; then echo " " ; else echo "-lordsdo11"; fi` -L/home/ronr/app/ronr/product/11.2.0/dbhome_1/ctx/lib/ -lctxc11 -lctx11 -lzx11 -lgx11 -lctx11 -lzx11 -lgx11 -lordimt11 -lclsra11 -ldbcfg11 -lhasgen11 -lskgxn2 -lnnz11 -lzt11 -lxml11 -locr11 -locrb11 -locrutl11 -lhasgen11 -lskgxn2 -lnnz11 -lzt11 -lxml11 -lasmclnt11 -lcommon11 -lcell11 -lskgxp11 -lgeneric11 -lcommon11 -lgeneric11 -loraz -llzopro -lorabz2 -lsnls11 -lnls11 -lcore11 -lsnls11 -lnls11 -lcore11 -lsnls11 -lnls11 -lxml11 -lcore11 -lunls11 -lsnls11 -lnls11 -lcore11 -lnls11 -lsnls11 -lunls11 -lsnls11 -lnls11 -lcore11 -lsnls11 -lnls11 -lcore11 -lsnls11 -lnls11 -lxml11 -lcore11 -lunls11 -lsnls11 -lnls11 -lcore11 -lnls11 -lasmclnt11 `cat /home/ronr/app/ronr/product/11.2.0/dbhome_1/lib/sysliblist` -R /opt/SUNWcluster/lib:/home/ronr/app/ronr/product/11.2.0/dbhome_1/lib:/opt/ORCLcluster/lib/ -Y P,:/opt/SUNWcluster/lib:/opt/ORCLcluster/lib/:/usr/ccs/lib/amd64:/usr/lib/amd64 -Qy -lc -lrt -laio -lposix4 -lkstat -lm /home/ronr/app/ronr/product/11.2.0/dbhome_1/lib/prod/lib/amd64/crtn.o -lpthread
    test ! -f /home/ronr/app/ronr/product/11.2.0/dbhome_1/bin/oracle ||\
    mv -f /home/ronr/app/ronr/product/11.2.0/dbhome_1/bin/oracle /home/ronr/app/ronr/product/11.2.0/dbhome_1/bin/oracleO
    mv /home/ronr/app/ronr/product/11.2.0/dbhome_1/rdbms/lib/oracle /home/ronr/app/ronr/product/11.2.0/dbhome_1/bin/oracle
    chmod 6751 /home/ronr/app/ronr/product/11.2.0/dbhome_1/bin/oracle

    Although this should be done offline, it also worked online on my test system, the test query shows

    SQL> select * from v$option where parameter = 'Real Application Testing';

    PARAMETER VALUE
    ------------------------- ----------
    Real Application Testing FALSE

    Only do this online at home, not in production

    2

    Opmerkingen tonen

  9. When dealing with AWR data using SQL, there is little doubt that you will end up writing SQL that has to compute the differences between values of two consecutive snapshots to get a delta value. Using delta values in a graph makes it a lot easier to spot changes and trends compared to viewing a slope, that also starts over again after every instance restart.

    In order to calculate with the correct values, the word consecutive is very important here. Just the fact that 2 rows ordered on begin_interval_time, dbid and instance_number are listed directly behind each other does not automatically mean that they are consecutive. For this article, consecutive also means that the snap_id’s differ exactly 1.

    A situation in which this has an effect is when a database instance is online for month’s and you converted older ranges of snapshots to base lines in order to build more history. In one of ‘my’ systems I do this for every Tuesday of the week. If in my AWR queries I do not take this into account I get funny values when I calculate the difference between to snapshots that are directly ordered behind each other but have quite a difference in time because normally the difference in time is 10 minutes and the difference in time between the last of Tuesday night and the first snapshot of the following Tuesday is a bit more than 10 minutes. For this reason I add an extra column to the dba_hist_snapshot view, in my queries.

    Since I often want to see how many days ago a sample was made, I also add a column ‘ago’ which tells me how many day’s ago the snapshot was made. With a few years of history this pays off. Because not all snapshots are of equal length, I also add the column interval_s that show the number of seconds the interval describes.

    To end, this is what I often use to start my AWR queries:

    WITH
    snaps AS
    (
    SELECT
    s.* ,
    SYSDATE - begin_interval_time ago,
    s.snap_id - row_number() over (partition BY s.dbid, s.instance_number, s.startup_time
    ORDER BY s.begin_interval_time) grp ,
    extract (DAY FROM (s.end_interval_time - s.begin_interval_time)*24*60*60) interval_s
    FROM
    dba_hist_snapshot s
    )
    select * from snaps order by begin_interval_time, dbid, instance_number;

    In the join with a metric view, almost for certain there will be a line that computes a difference between values using a lag or lead function:

       , h.total_waits - lag (total_waits) over (partition by s.dbid, s.instance_number, s.startup_time, s.grp, h.event_name
    order by s.begin_interval_time) d_total_waits

    A complete working example gives:

    WITH
    snaps AS
    (
    SELECT
    s.* ,
    SYSDATE - begin_interval_time ago,
    s.snap_id - row_number() over (partition BY s.dbid, s.instance_number, s.startup_time
    ORDER BY s.begin_interval_time) grp ,
    extract (DAY FROM (s.end_interval_time - s.begin_interval_time)*24*60*60) interval_s
    FROM
    dba_hist_snapshot s
    )
    , measurements as
    (
    select s.*, h.wait_class, h.total_waits, h.time_waited_micro
    , h.total_waits - lag (total_waits) over (partition by s.dbid, s.instance_number, s.startup_time, s.grp, h.event_name
    order by s.begin_interval_time) d_total_waits
    from dba_hist_system_event h
    , snaps s
    where h.event_name = 'db file async I/O submit'
    and h.snap_id = s.snap_id
    and h.dbid = s.dbid
    and h.instance_number = s.instance_number
    )
    select * from measurements
    order by begin_interval_time, dbid, instance_number;

    Filtering to get specific time ranges can be done in the snaps view, filtering to get a specific event can be done in the measurements view. The next step is to take care of varying length of snapshot intervals. Here comes the interval_s column into play:

    WITH
    snaps AS
    (
    SELECT
    s.* ,
    SYSDATE - begin_interval_time ago,
    s.snap_id - row_number() over (partition BY s.dbid, s.instance_number, s.startup_time
    ORDER BY s.begin_interval_time) grp ,
    extract (DAY FROM (s.end_interval_time - s.begin_interval_time)*24*60*60) interval_s
    FROM
    dba_hist_snapshot s
    )
    , measurements as
    (
    select s.*, h.wait_class, h.total_waits, h.time_waited_micro
    , h.total_waits - lag (total_waits) over (partition by s.dbid, s.instance_number, s.startup_time, s.grp, h.event_name
    order by s.begin_interval_time) d_total_waits
    from dba_hist_system_event h
    , snaps s
    where h.event_name = 'db file async I/O submit'
    and h.snap_id = s.snap_id
    and h.dbid = s.dbid
    and h.instance_number = s.instance_number
    )
    , bytime as
    (
    select m.*
    , m.d_total_waits / interval_s waits_per_s
    from measurements m
    )
    select * from bytime
    order by begin_interval_time, dbid, instance_number

    If all intervals are of equal length, a graph based on d_total_waits will look very similar as based on wait_per_s. In this case it is pretty clear when I run a backup ....graph

    Around instance startup times, there will certainly be a variation in interval_s.

    I am sure this can be written cleaner and maybe even meaner but for my simple dba purposes this does what I need and take care of most things I need to take care about. I hope it also helps others. Thanks Rob for the explanation about getting the grp.

    0

    Een opmerking toevoegen

  10. The blog message from Wim Coekaerts was a welcome invitation to go an try to play with dtrace on Linux. The directions to get it all up and running are pretty clear but still …. maybe it can be made a little easier.

    Wim already told to enable the beta repo, I will repeat that here:

    cd /etc/yum.repos.d/
    wget http://public-yum.oracle.com/beta/public-yum-ol6-beta.repo
    sed -i s/enabled=0/enabled=1/g public-yum-ol6-beta.repo

    check with

    yum repolist
    Loaded plugins: security
    repo id repo name status
    public_ol6_UEKR3_latest Unbreakable Enterprise Kernel Release 3 for Oracle Linux 6Server (x8 53
    public_ol6_latest Oracle Linux 6Server Latest (x86_64) 24,323
    uek3_beta Unbreakable Enterprise Kernel Release 3 for Oracle Linux 6 (x86_64) 188
    repolist: 24,564
    yum update

    will install the UEK3 kernel, if it is not already present. Make sure that you boot using this new UEK3 kernel before installing Dtrace. Note: don't forget to enable the uek3_beta repository, otherwise, there won't be a Dtrace for you, at this moment in time.

    yum install kernel-uek-devel-3.8.13-16.2.1.el6uek.x86_64
    to enable the compilation of the vm support tools.

    As of this writing, the kernel version where dtrace is included is 3.8.13-16.2.1.el6uek.x86_64 If you have this exact kernel version running dtrace can be installed with

    yum install dtrace-utils

    Dtrace is very powerful and having it limited to only the root user is a bit of a pity. So I am going to enable access to it for other users as well by dedicating a group to dtrace enabled users. The group for this I call dtrace. I will add the dtrace group to the users that I want to give access to Dtrace.

    groupadd dtrace
    usermod -a -G dtrace myuser

    This alone is not enough. Dtrace controls access using a device file that is generated in /etc/dtrace/, when the dtrace module is loaded. So this is a dynamic thing that vanishes when the system stops or when the dtrace module is unloaded. Just changing the permissions or ownership on this file is not going to last very long. After each reboot, the change has to be applied again. Here comes udev to rescue, again.

    echo 'kernel=="dtrace/dtrace", GROUP="dtrace" MODE="0660"' >/etc/udev/rules.d/10-dtrace.rules

    Now we are mostly done but not complete yet, the modules are not yet auto loaded. Add the lines to rc.modules:

    cat <<eof >>/etc/rc.modules
    /sbin/modprobe dtrace
    /sbin/modprobe profile
    /sbin/modprobe sdt
    /sbin/modprobe systrace
    /sbin/modprobe dt_test
    eof
    chmod 755 /etc/rc.modules

    Now, after a reboot, the modules are loaded, the dtrace helper devices are created, assigned correct group ownership and permissions so Dtrace must be usable.

    dtrace -l|wc -l
    642

    For Linux there still is some work to do, on MacosX (10.9) this gives:

    dtrace -l|wc -l
    234004

    errors

    If you encounter an error message like dtrace: failed to initialize dtrace: DTrace requires additional privileges, check the permissions of /dev/dtrace/dtrace. The dtrace user must have write permission on this file. The doc says the permission is needed on /dev/dtrace/helper but my observation is slightly different.

    An other error that initially hit me was dtrace: failed to initialize dtrace: DTrace device not available on system My kernel at that moment was 3.8.13-16.2.2.el6uek.x86_64 and not 3.8.13-16.2.1.el6uek.x86_64. For the newer kernel I could not find the dtrace-util package. After rebooting using the correct kernel the error persisted, this time because the modules were not loaded.

    future

    I am sure there will be more development for Dtrace. It looks a lot like the use of Dtrace will be restricted to ULN users. Let's hope that Oracle changes it's mind on this and makes Dtrace usable for all. Also read http://bdgregg.blogspot.nl/2013/09/dtracetoolkit-0xx-mistakes.html

    0

    Een opmerking toevoegen

Laden