1/16/2014

FBScanner 3.6.1 is released

New version of FBScanner (3.6.1) is released. FBScanner is an unique tool to trace and audit Firebird (1.0 - 2.5) and InterBase (4.0 - XE3) SQL traffic (connections, transactions, queries), it's very efficient tool for performance optimization and SQL bug tracking.
FBScanner works like a proxy, and enables transparent data capture and analysis:
FBScanner is Firebird performance optimization tool
FBScanner can be used for the following tasks:
  • Find queries with bad performance (i.e., slow queries with high prepare and execute times, queries with bad plans)
  • Track transactions usage
  • Real-time view of connections, users and transactions (connections can be killed)
  • Retrospective analysis of SQL traffic
  • and more - see presentation here.
FBScanner has 2 licensing options:
  • Per server - FBScanner can be used on the single server or computer to track Firebird traffic.
  • Unlimited ISV subscription - FBScanner can be installed on any number of computers during the 1 year for use with the specific software application from the single independent software vendor (ISV).
All users of FBScanner 3 already have FBScanner 3.6.1 in IBSurgeon Deploy Center.
User of FBScanner 2.x can upgrade to FBScanner 3.6.1 with 50% discount - please contact upgrade@ib-aid.com


Download now:
Demo version of FBScanner can be found here:
FBScanner Demo version
Please look at the FBScanner User Guide to get more information about its capabilities:
FBScanner User Guide

11/12/2013

Firebird Tour 2013: detailed schedule for seminar “All About Transactions”

Firebird Tour 2013: seminar “All About Transactions”

Overview

#
Topic
Speaker
Time
1
Overview of Firebird development
Vlad Khorsun
30
2
Overview of transactions concept and its parameters.  Examples of transactions parameters and their usage.
Alexey Kovyazin/Pavel Cizar
45
3
Transaction in Firebird: peculiarities and special parameters.  Transaction inventory page, versions and how transactions work in multi-generation database.
Dmitry Kuzmenko
60

Coffee-Break

15
4
Inside Firebird transactions: what happen inside engine during transactions’ start, commit, rollback. What is retaining, autocommit, how transactions work with GTT. Memory consumption for transactions processing.
Vlad Khorsun
60
5
Transactions interaction and best practices for application development. Conflicts between transactions.
Dmitry Kuzmenko
60

Lunch

60
6
Garbage collection mechanism and sweep in details
Vlad Khorsun
45
7
Transactions management in various drivers (Delphi, .NET, Java, etc): parameters and approaches.
Dmitry Kuzmenko/Jiri Cincura
45

Coffee-Break

15
8
Diagnosing transaction management problems: most prevalent problems, how to catch and fix them.
Alexey Kovyazin
60
9
Round table

45

Detailed plan

   1.       Overview of Firebird development
a.       New features and details of Firebird development,
b.      Plan for next releases
   2.       Overview of transactions concept and its parameters.  Examples of transactions parameters and their usage.
a.       Historical overview of transactions: locking engines, dirty read, why Firebird has only snapshot и read_committed
b.      Transaction as logical operation: bridge between business logic and database implementation
c.       What is ACID
d.      Isolation levels: standard levels and Firebird levels
e.      read only\write
f.        wait and nowait
g.       Examples of transactions with various isolation levels, read-only/write and wait/nowait, from the client point of view
   3.       Transaction in Firebird: peculiarities and special parameters.  Transaction inventory page, versions and how transactions work in multi-generation database.
a.       Full syntax of SET TRANSACTION and TPB parameters
b.      Some undocumented parameters
c.       Who create transactions
                                                               i.      Application
                                                             ii.      System transaction (tx #0)
                                                            iii.      Garbage collector
                                                           iv.      Autonomous transactions
                                                             v.      Triggers ON CONNECT\ON DISCONNECT
d.      Versioning basics
e.      How versions are being created
f.        States of transaction in Firebird (active, committed, rolled back, limbo)
g.       Visibility of record versions in transactions
h.      Key differences between Read Committed and Snapshot
i.         Triggers for transactions start and stop
j.        Context variables at transaction’s level
k.       GTT and read-only transactions

   4.    Inside Firebird transactions: what happen inside engine during transactions’ start, commit, rollback.
a.       What happens at transactions’ start
                                                               i.      Global transaction markers (OAT, OST, OIT) and their meaning
                                                             ii.      Autosweep
b.      What happens at transaction commit
                                                               i.      How DWF works
                                                             ii.      DDL queries
                                                            iii.      Dispatching events
                                                           iv.      Flush of dirty pages
                                                             v.      TIP processing
c.       What happens at rollback
                                                               i.      Standard rollback with cache flush
                                                             ii.      Forces rollback without cache flush
                                                            iii.      Rollback transactions with undo-log
                                                           iv.      Rollback without undo-log
d.      What is retaining
e.      Auto-commit (does anybody know about it?)
f.        Managing memory for transactions
                                                               i.      Temporary BLOBs
                                                             ii.      Undo-log
g.       Local TIP and related issues
   5.       Transactions interaction and best practices for application development. Conflicts between transactions.
a.       Transactions interactions types
b.      Templates for transaction usage in application development
c.       Scenarios breakdown: Readers and Readers, Readers and Writers, Writers and Writers
d.      Best practices
                                                               i.      Dictionaries
                                                             ii.      Splitting read and write transactions
                                                            iii.      Locking record with empty Update
                                                           iv.      Locking with SELECT WITH LOCK
                                                             v.      Reports
                                                           vi.      Serialization: how to build effective queues

   6.    Garbage collection mechanism and sweep in details       
a.       Why we need sweep?
b.      How sweep works
c.       Types of the sweep: cooperative, background, background
d.      Differences between sweep and garbage collection
e.      Autosweep: pro and contras
f.        Examples of application design mistakes which lead to excessive garbage

   7.   Transactions management in various drivers (Delphi, .NET, Java, etc): parameters and approaches.
a.       What are direct and non-direct client drivers?
b.      BDE, dbExpress, ODBC, .Net
c.       IBX, FIBPlus, FireDAC
d.      Transactions management in applications: examples
e.      How to manage transactions lifetime in various drivers
f.        How to implement edit dialogs in the effective and safe way
g.       Commit retaining

   8.   Diagnosing transaction management problems: most prevalent problems, how to catch and fix them.
a.       What we can see at gstat -h
b.      OIT, OST, OAT, NEXT  - their combinations and what they mean
c.       When garbage collection does not work?
d.      Transaction markers dynamics analysis with IBTM. Catching sweep moments.
e.      How to determine quantity of garbage versions
f.        How to find long transactions with MON$ tables
g.       Transactions monitoring with MON$ and TraceAPI.
h.      Legacy applications: workarounds for error in transactions management

   9.   Round table: questions and answers

9/24/2013

Answers for 3 transactions questions

So, it's time for the answers.

The question 1 is the easiest:

1.       In some transaction there were 3 inserts made. The third INSERT raises exception. Is it possible to commit this transaction?              
                [ ] yes, [ ] no

Answer is Yes, of course. An ability to commit or rollback transaction does not depend on insert operations which were performed in the frame of transaction. Of course, usually application logic is built in the way to commit only successful set of operations, but it's not a rule. 

The second question is a bit more tricky:

2.       In some table there were 10 records. 3 simultaneous transactions were started and then in each transaction 1 record was inserted. In transaction 3 query SELECT COUNT(*) FROM Table was executed. The result will be:      
[ ] 10, [ ] 11, [ ] 12, [ ] 13

In this question you need to take an account that transactions are not committed. As you know, all changes in the frames of non-committed transaction are visible only inside this transaction, so the answer will be 11.


The third question is devoted to the generators concept of Firebird engine.


3.  Initially we have generator with name id, its value is 10. We have simultaneous transactions A and B. In A executed gen_id(id,1), in B gen_id(id,1). Then A was commited and B was rollbacked. What value is in id?      
                [ ] 10, [ ] 11, [ ]12

Generators (or sequences) work outside the frames of transactions. It means that commit or rollback of transactions does not affect the resulted value - i.e., generator ID will be increased both in A and B, and correct answer is 12.

Did you answer all these questions correctly? I guess so, since they are very easy. 
There are more interesting topics related with transactions. For example, do you know that each database in Firebird has a limit of transactions which can be performed between backup/restore cycles? And, more interesting, that unused transactions occupy space in the database, which is released only after backup/restore, and there could be up to 700Mb of such space in the worst case?

If you are interested, consider to visit Firebird 2013 Tour  seminar "All About Transactions" in Siegburg, Germany, November 22, 2013.

9/23/2013

3 questions about transactions every Firebird developer should answer correctly

Hi All,

As you probably know, Firebird Project will have a seminar, devoted to transactions topic, in Siegburg (Germany), November 22, 2013. More details: http://firebirdsql.org/en/firebird-conference-tour-2013/
Many developers think that transactions are straightforward and easy thing to understand and use, so probably there is no reason to devote 1 day seminar to discover transactions mechanism in Firebird in great details. Is it so?
Here I offer 3 simple questions about transactions which every Firebird developer should answer. Please take a look and choose the correct answer:


1.       In some transaction there were 3 inserts made. The third INSERT raises exception. Is it possible to commit this transaction?              
                [ ] yes, [ ] no
2.       In some table there were 10 records. 3 simultaneous transactions were started and then in each transaction 1 record was inserted. In transaction 3 query SELECT COUNT(*) FROM Table was executed. The result will be:      

[ ] 10, [ ] 11, [ ] 12, [ ] 13

3.  Initially we have generator with name id, its value is 10. We have simultaneous transactions A and B. In A executed gen_id(id,1), in B gen_id(id,1). Then A was commited and B was rollbacked. What value is in id?      
                [ ] 10, [ ] 11, [ ]12


So, take your time and choose correct answers. Tomorrow we'll consider the questions and their answers.

4/11/2012

Update 1 for FBDataGuard 2.7

Update 1 for FBDataGuard 2.7 is available for download (archive for LinuxUser Guide). It replaces the previous version of FBDataGuard 2.7. This update contains important stability improvements (for environments with more than 200 concurrent users) and several bugfixes. 

Please upgrade your FBDataGuard 2.7 installations as soon as possible. This is a minor upgrade, so instead of full reinstallation you can stop FBDataGuard 2.7 service, replace dataguard.jar (from new dataguard27.zip) and start it again. 

Do not hesitate to ask any questions: support@ib-aid.com.

1/20/2012

DBInfo - new IBSurgeon's free product



IBSurgeon DBInfo is designed to visualize internal structure's statistics, so administrators or developers can view how much data, index, blob or other kind of information your Firebird or InterBase database stores. 


Frequiently Asked Questions

1. What is the purpose of DBInfo?


This tool allows you to
  • view how much data, index, blob or other kind of information your InterBase or Firebird database store
  • detect some database corruptions
  • look at database page size, overall file size, database format
  • have some fun watching dynamically changing % indicators.

2. What InterBase and Firebird versions are supported?


Any from InterBase 4.0 to the latest InterBase and Firebird versions. DBInfo supports the following database formats:
  • 8.x - InterBase 4.x
  • 9.x - InterBase 5.x
  • 10.0 - InterBase 6.x, Firebird 1.0
  • 10.1 - Firebird 1.5.x
  • 11.x - InterBase 7.0/7.1/7.5
  • 11.x - Firebird 2.0/2.1/2.5
  • 11.x - Firebird 2.x
  • 12.x - InterBase 2007
  • 13.x - InterBase 2009
  • 15.x - InterBase XE

3. Do I need to have InterBase or Firebird server running?


No. DBInfo can check database file by itself, without server, and it does not need client library also.

4. Can I check for database contents on remote server?


A. Yes and no. Since DBInfo opens database file by itself, i.e. use direct access to database, you can check for database contents of database on remote server only if this database file is being shared on network.

5. May I use DBInfo if my database is on Linux (or other Unix)?


No, DBInfo is a Windows Win32 tool, so you may only check database files that can be accessed as files from Windows - locally or at shared folders.

6. Can DBInfo corrupt my database?


Never. DBInfo only reads the database file, and never writes to it. The open mode of the database file is read-only in any case.

7. May I check database contents while InterBase or Firebird is working with it?


Yes. DBInfo can open the database file even when InterBase or Firebird works with it. But, since reading the whole database file affects disk performance, be sure that at the moment your database is not at high load by applications. 
Also, if you will check the database when the database is being modified by InterBase or Firebird, DBInfo may show warning that database is corrupted, which may be wrong.

8. What is the purpose of the option "Do not use Windows file cache"?


DBInfo can scan database file with two modes - using Windows cache or not using it. Usually when you copy file on Windows, it goes to the file cache. And, file cache is being allocated from physical memory. So, the bigger is file, the more amount of file cache is used by Windows. This can cause applications and other data to being moved to virtual memory, and slowdown computer performance. 
For example, for database files larger than 500mb you could notice contiguous slowdown of DBInfo while scanning database file. Thus, we decided to turn off using cache in DBInfo by default. This allows to scan 1-10-100 gb databases without performance loss for DBInfo and operating system itself.

9. Why percentage of blob pages is marked by that color?


The colors is used to paint percentage bars does not have any meanings, except for bad pages found that is marked as red. This is the only case when we pays your attention to the damaged pages found in database.

10. What I can do if DBInfo found that my database is corrupted?


The better way is to immediately check the database by gfix and IBFirstAid Diagnostitian, and send full log to support@ib-aid.com. We will be able to help with Database Repair Services or you can buy gull version of FirstAID.

11. What if I want to see how much space is taken for each table and index in database?


In this case we suggest you to use our IBAnalyst tool, that not only can show you detailed information about your database, but also can give some notes and suggestions about performance optimizations