Hajautetut tietokannat Talvi 2016 Vastaukset Laskuharj. 5. CS-E4630 Distributed Databases Winter 2016 Answers Tutorial 5 1/9

Hajautetut tietokannat Talvi 2016 Vastaukset Laskuharj. 5. CS-E4630 Distributed Databases Winter 2016 Answers Tutorial 5 1/9 Sivupalvelimet sekä yhteislevyjärjestelmät Page Servers and Shared Disk Systems Vastaukset useisiin kysymyksiin löytyvät myös artikkelista: Crash recovery in client-server EXODUS, Carey et al, Proceedings of the 1992 ACM SIGMOD international conference on Management of data, Pages 165-174. linkistä http://dl.acm.org/citation.cfm?id=130283.130312 The answers to many of these questions can be found in the article above, with the online link. Chapter 4 in the English study-guide should also help, in the Finnish material, this is section Sivupalvelin- ja yhteislevyjärjestelmät [0] [a] Explain the division of labor in a C/S system. The server, being more robust and powerful than the client, is responsible for managing the database on disk along with its log. The Server is multi-threaded and handles locking, page allocation as well as recovery of the database in case of failure. The client is typically less powerful in terms of memory and CPU than the server, although the total power of the clients combined is usually greater than at the Server. The normal data updates and index maintenances are performed by clients. The Client requests the pages it needs from the Server, performs the required updates on the obtained data pages (after logging the changes to its log kept in memory) and then sends the log records to the server, usually together with the modified data pages. A C/S system is not to be confused with a file server system, which is more similar to a centralized DB. In a true C/S environment, the DB is stored on a machine which is separate from those machines that run user applications or parts of user applications. [b] What is query shipping? What is data shipping? Query is shipping is about sending the SQL-query to the server, to be processed over there. The result set is then shipped to the client. With data shipping, clients request the required data from the server and the transactions are run locally. This makes caching possible. [c] Mikä voi romahtaa C/S järjestelmässä? What can crash in a C/S system? Crashes in the client-server environment can be classified into a client crash,

Hajautetut tietokannat Talvi 2016 Vastaukset Laskuharj. 5. CS-E4630 Distributed Databases Winter 2016 Answers Tutorial 5 2/9 a server crash, or a crash in both client/server. [d] What are the benefits of caching data at the client? A client might need to read a data item repeatedly, read and write the same data item or run multiple transactions on the data item. Caching a data item allows the client to run the transactions locally (without contacting the server). It can then write back the final value when it commits. In a data shipping client-server system each client has a cache which is basically a buffer pool to place the pages as they are fetched from the Server. Updates are primarily done at the client side, while the Server is the one which keeps a stable copy of the database and maintains its log. [e] Give an advantage obtained with the data server with respect to a query shipping server. Data-shipping architectures have the potential advantage of avoiding bottlenecks at the server (less load on the server) by exploiting the processing power end memory of the client machines. This is important for performance, since today s clients can have processing power and memory similar to the server s. [f] Give a brief but concise definition for: (i) dataserver, (ii) Client s WAL-policy (iii) inter-transaction caching. data server: tietokantapalvelin, joka välittää asiakkaalle tietoalkioita pyynnöstä. Yleensä kyseessä on sivupalvelin (lähetysyksikkö on fyysinen sivu) tai oliopalvelin, jolloin lähetysyksikkönä on looginen olio. A database server ships data items on request to clients. A data server usually operates/ships either in physical units (e.g. pages hence a page-server) or in logical data units (e.g. objects hence an object server). Client s WAL-policy: C/S ympäristössä asiakkaan täytyy lähettää palvelimelle sivua koskevat lokitiedot *ennen* varsinaisia likaisia sivuja. In a C/S environment, the Client must send the Server the log records for a modified page *before* sending the actually modified page(s). Transaktiorajat ylittävä puskurointi: puskurointikäytäntö, jossa asiakas voi pitää aineistoa puskurissaan, vaikka transaktio olisi jo sitoutunut. Monimutkaisempaa kuin transaktion sisäinen puskurointi, sillä tarvitaan protokolla hajautettujen lukkojen hallintaan. Transaktion lokitiedot tulee kuitenkin lähettää palvelimelle. Inter-transaction caching: A caching policy where the client can keep data at its buffer pool (=cache) even after the transaction has committed. The log records for the transaction must be sent to server at commitment though. [1] [a] Even though the No-Steal policy is simpler for recovery (only committed pages get written to the physical DB), the STEAL-policy is used for performance reasons. What does it allow in practice? With STEAL, any page can be written to the disk, so STEAL can be used to create space in the buffer for other transactions. This is especially helpful if the transaction being run is a long one and needs to access many pages before commitment. If two transactions are accessing the same page, as soon as one transaction needs to commit, the other transaction s changes can be written to disk even though that transaction is not yet ready to commit and so the transaction that was going to commit can proceed immediately. [b] What two rules were required by WAL protocol used in ARIES? (i) Before writing a dirty page back to disk, must make sure that all the before-images (value before update) get written to the physical, stable log. This helps the UNDO operation. (ii) When a transaction commits, all its after-images (value after update) must have been written to the physical, stable log. This helps the REDO operation.

Hajautetut tietokannat Talvi 2016 Vastaukset Laskuharj. 5. CS-E4630 Distributed Databases Winter 2016 Answers Tutorial 5 3/9 Käytännössä siis transaktio on sitoutunut kun sen sitoutumismerkintä on viety pysyvään lokiin (varsinaiset sivujen kirjoitukset tietokantalevylle voivat tapahtua myöhemmin). For all practical purposes, a transaction is considered to have committed when its commit record has been written to the stable log (the actual pages can be written to the database later). [c] Where is the log located in a C/S system using page-server? Do the clients do any logging? When is the page at the client referred to as modified( dirty )? The log is in a disk system that is attached to the database server only. Clients log operations into memory, not to disk, their logs are sent to the server. The page at the client is dirty when it differs from the page at the server. [d] In a C/S system using page-server, what is the main task of the DB process operating at the Client? And the main task for the DB process operating at the server? Asiakas-tietokantaprosessi muuntaa sille tulevat paikalliset kyselyt tiettyjen sivujen pyynnöiksi ja on vastuussa näiden sivujen tuomisesta palvelimelta asiakkaan muistiin. Palvelimen tehtävänä on tarjota asiakkaalle tämän niitä pyytäessä ajan tasalla olevat datasivut. Palvelinprosessi omistaa sivuissa välitetyn aineiston ja on viime kädessä vastuussa siitä, että datan oikeellisuus säilyy. Each client DBMS process translates the requests of the local query into a request dealing with specific pages and is responsible for bringing those pages from the Server into the client s memory. It is the responsibility of the server DBMS process to provide the client with the most recent committed values for the requested data pages. The Server DB process is the owner of the shipped pages and is responsible for maintaining data integrity. [e] REVIEW Fill in the table using check-marks and rate the scalability (low, medium, high). Shared? Own Memory Own Disks Scalability Shared memory/shared everything Own Database Management Software LOW Shared Disk X X MEDIUM Shared Nothing X X X HIGH [f] What are the two main tasks of buffer coherency control in shared-disk system? i) It must detect when a page on disk becomes invalid (i.e. is no longer clean as a result of being updated in the buffer of one node) ii) It is responsible for providing the node with the most-up-to-date version of the page, meaning when a node needs a new version of a given page, the buffer coherency control must know from where to get it. The owner of the page is the only node authorized to write a dirty page back to the disk and if requested, must provide other nodes with the up-to-date version of that page. [g] In an Shared-Disk environment, if a modified page is cached in one node (referred to as the owner), then a different node requiring access to that page must get the current/latest(=nykyversio) version of the page from the owner. Special locks (so-called P-locks) are used to help in detecting that a cached page in a particular buffer pool is not the latest version of the page. When a transaction T 1 updates a record on page Pi in node N1, the transaction updates the page_lsn of that page Pi. When T 1 then commits, the current page_lsn of Pi is sent to the Lock Manager (LM). The LM registers the page_lsn in the P-lock entry for Pi, before unlocking the page s usual logical locks. When T 2 in N 2

Hajautetut tietokannat Talvi 2016 Vastaukset Laskuharj. 5. CS-E4630 Distributed Databases Winter 2016 Answers Tutorial 5 4/9 locks a record in Pi, it requests the page_lsn for page Pi. When LM returns this page_lsn as part of granting the required lock=(osana lukon myöntämistä), Node N 2 can then detect if its cached/buffered =puskuroitu version, if any, of Pi is not current. If Pi at node N 2 is not current, then the Lock Manager will assist node N 2 in getting the current version of page Pi from the owner of that page =sivun omistajalta, Node N 1. [h] A global lock table stored at the Server contains what information? The global lock table stores at the Server the locks for each client as the tuple (c, x, m, d) where x is the name of the lock, m is the lock-type (e.g write or read lock), d its duration such as commit-duration, shortduration(=kestoaika) and c is an identifier for the client. [2] In a page-server system (C/S model), transaction T for client c requests an S lock for key value x. In which situations can the lock be granted locally without for c having to communicate with the server? Suggest a way for reducing the communication between client and server due to the requesting of S-locks. (Tehtävään on useita ratkaisuja.) Ensinnäkin, jos asiakkaan c lukkotaulussa on jo S-lukko jollekin muulle c:n transaktiolle, asiakas voi myöntää saman (tai pienemmän) lukon kommunikoimatta palvelimen kanssa. Seuraavassa eräs keino, jolla S-lukkojen hallinnan saa lähes kokonaan asiakkaan sisäiseksi. Muutetaan lukkojen varausta seuraavasti: Kun palvelinprosessi lähettää sivun p asiakkaalle c, samalla c:lle annetaan oikeus varata S-lukkoja kaikkiin sivun p tietueisiin, kunhan niihin ei ole varattu X-lukkoja globaalissa lukkotaulussa. X-lukot saa selville niin, että palvelin lähettää sivun p mukana luettelon kaikista sivun p tietueisiin varatuista X-lukoista. ( X-lukitut tietueet voidaan merkitä suoraan sivun p asiakkaalle annettavaan kopioon metatiedolla, joka kertoo, että asiakas ei saa lukea niitä, paitsi hakemalla palvelimelta globaali S-lukko). Jos palvelin hakee sivun p takaisin c:ltä, c:n täytyy samalla kertoa palvelimelle kaikista sen paikallisesti varaamistaan voimassaolevista S-lukoista tämän tiedon palvelin voi tallettaa suoraan globaaliin lukkotauluun. Here s one way to make the control of S-locks almost fully internal to the client. Basic idea: give the client the right to place on the page any locks which are not conflicting. Change the lock acquisition as follows: when the server process sends page p to client c, also grant c the right to place S-locks on all records on page p, as long as there are no X-locks placed on them in the global lock table. The existing X-locks can be determined by having the server send with page p a list of all X-locks placed on records of page. (The X-locks can be marked right into the copy of page p given to the client using metadata that says the client is not allowed to read them, except by getting a global S-lock from the server). If the server requests back the page p from client c, then c must also inform the server of all the local locks it has placed on the page. The server can save this information directly into the global lock table

Hajautetut tietokannat Talvi 2016 Vastaukset Laskuharj. 5. CS-E4630 Distributed Databases Winter 2016 Answers Tutorial 5 5/9 [3] Assume that in a three-tiered model, a) the presentation services at the client machine (e.g. the browser) crashes during a transaction. What happens in the remaining servers? What if instead of a crash, the machine in question just temporarily loses its network connection to other servers? An example of a product that supports the three-tiered model is SAP/R3, which is used by many of the large production and retail companies. Typically, the application layer is made up of one or more application servers which run on powerful servers. The database layer runs on top of the actual database product (MySQL, DB2, etc.) which will be in a single powerful database server. In case the database is accessed via a smartphone, the mobile would become in fact the presentation layer. MAIN IDEA: ApplicationLayer Notices PresentationLayer Inactive aborts transaction: If PresentationLayer did not crash, (network error) ApplicationLayer informs it that transaction aborted Luultavasti sovelluspalvelimessa on aikakatkaisu, joka keskeyttää transaktion (mahdollisesti kysyttyään esityspalvelulta lupaa), jos esityspalvelulta ei pitkään aikaan tule transaktioon liittyviä pyyntöjä. Sovelluspalvelin siis lopulta pyytää tietokantapalvelinta keskeyttämään transaktion. Mikäli esityspalvelu ei romahtanutkaan, vaan kysyy myöhemmin transaktion tilaa tai tekee siihen liittyviä pyyntöjä, sovelluspalvelin ilmoittaa että transaktio on keskeytynyt. (Aikakatkaisu voi olla sovelluspalvelimen sijasta myös suoraan tietokantapalvelimessa, ja saattaa liittyä myös palvelinten välisen verkkoyhteyden ylläpitoon.) In the application services there is most likely a timeout that interrupts (possibly asking permission from the presentation services first) the transaction, if no requests for the transactions emanate from the presentation services for a long time. The application service thus finally asks the database server to abort the transaction. In case the presentation services did not crash, and asks instead the state of the transaction or processes requests attached to it, the application services informs that the transaction has aborted. (The timeout can be implemented directly in the database server instead of the application services and may be part of maintaining the network connection between the different services).

Hajautetut tietokannat Talvi 2016 Vastaukset Laskuharj. 5. CS-E4630 Distributed Databases Winter 2016 Answers Tutorial 5 6/9 b) the database serve crashes MAIN IDEA: ApplicationLayer Notices DatabaseLayer Inactive assumes abort of transaction, notifies PresentationLayer If DatabaseLayer did not crash, (network error) ApplicationLayer can request abort of transaction Sovelluspalvelin huomaa tietokantapalvelimen romahduksen viimeistään siitä, että tietokantapalvelin ei vastaa määräajassa johonkin sovelluspalvelimen tekemään pyyntöön. Nyt sovelluspalvelin olettaa että transaktio on keskeytynyt ja ilmoittaa tästä esityspalvelulle. Mikäli tietokantapalvelin ei romahtanutkaan, vaan verkkoyhteys palaa myöhemmin, sovelluspalvelin pyytää tarvittaessa transaktion keskeyttämistä. The application services notices the crash of the database server by latest when the database server doesn t answer within a time limit to some request made by the application services. Now the application services assumes the transaction has aborted and notifies the presentation services about this. If the database server did not crash, and the network connection gets re-established, the application services will request for the transaction to be aborted if necessary. [4] [a] True or False? In shared disk environment using the Fast scheme, the page being shipped might contain updates from more than one node. True: After a modified page is written to disk, it becomes clean again. In the simple and medium scheme, pages are written to disk before the requesting node can use the page. In the Fast scheme this is not the case. [b] What is a fuzzy checkpoint? A fuzzy checkpoint means that the data pages are not written to disk (only so-called Modified Pages Table and Transactions table). ARIES assumes most checkpoints are fuzzy as they are faster than complete checkpoints. [c] What is the purpose of REDO phase in ARIES? What about the UNDO-phase? To REDO every page that was modified and so make sure that all updated pages are actually on disk. (the system will be in the same state as before the crash). The UNDO phase will undo the effects of those transactions that had not committed at the time of the crash. [d] List two ways in which ARIES can identify a page as being modified when doing recoveryi) BEFORE taking the checkpoint, ARIES has marked the page as updated and so this information gets written to disk in the Fuzzy Checkpoint. ii) The page may not be in the modified pages table found in the checkpoint (because ARIES has just taken a checkpoint), but AFTER the checkpoint, there are log records referring to updates for the page and these logs records have been taken to disk. So on recovery, ARIES can deduce that the page was modified before the crash. [e] What would happen if ARIES could not correctly identify a page as being modified? That page would be skipped in the REDO-phase, meaning there would be no guarantee that its changes have actually gone to disk. Since in ARIES all pages belonging to an uncommitted transaction get UNDONE, trying to UNDO changes that are not on the database disk could in the worst case corrupt the database. Even if there is corruption for the, but the transaction committed before the crash, the updates will be lost, so the database will end up in a non-consistent state.

Hajautetut tietokannat Talvi 2016 Vastaukset Laskuharj. 5. CS-E4630 Distributed Databases Winter 2016 Answers Tutorial 5 7/9 [f] Explain how such a situation might arise in a C/S environment. The Client updates page p and logs the changes in its memory-log. The Client sends these logs for page p to the Server. The Server then takes a fuzzy-checkpoint and writes the modified pages table into this checkpoint. However, at this point, page p is clean (not modified) according to the Server. The Client is committing the transaction, no longer needs the page and so sends page P to the Server. Page p arrives from the Client after the Checkpoint has been taken. The Server writes a Commit record to the log, so the transaction is considered to have committed according to WAL, even though the data page p received from the Client is not yet on disk. The Server CRASHES and the data in page p is lost by the Server. So the logs for page p are received BEFORE the fuzzy checkpoint, and the actual data page p is received AFTER the fuzzy checkpoint. Then the Server crashes. When the Server recovers and starts ARIES: The page p will not be in the Modified-Pages Table ( Dirty Pages Table ) that was stored during the checkpoint, because the page was received AFTER the checkpoint was taken. There are no logs referring to updates to page p in the log in the disk *after* the checkpoint, because the log records were received from the client BEFORE the checkpoint was taken so ARIES at the Server is unaware that there are missing updates for the page. A simple solution to avoid missing any dirty page is for the Server to record in its log that a modified page has arrived from the Client.

Hajautetut tietokannat Talvi 2016 Vastaukset Laskuharj. 5. CS-E4630 Distributed Databases Winter 2016 Answers Tutorial 5 8/9 [5] Review [a] Given n = 6 and p = 4. The size of the Write Quorum q must be the majority of the sites or q> n/2, so q= 4. Suurin määrä pisteitä, jotka voivat romahtaa jollekin p:n ja q:n arvoille saadaan yhtälöstä: R = n- max{p,q}. The max number of sites than fail for any given p and q is given by R= Robustness=( Romahduksien sietokyky )= n max{p, q}. Here n max{p, q} = 6 4 = 2. So only 2 sites can fail. In the figure, n=6. A possible Write quorum is shown in blue with q=4, a possible Read quorum is shown in gray with p=4 also. The two sites marked in red can fail. If n=6 and q=5, will this change anything? Here assume that p is not given. The minimum value for p is now p=2 and if q= 5 then only one site can fail. Robustenss =n-max{p,q}=6-5=1. Note that reducing from p 4 to 2 makes Reading easier: only two sites need to obtain a read lock on the data item. [b] Which transactional property is enforced by distributed commit protocols? Explain briefly. Distributed commit protocols enforce the atomicity property. The idea is that all or none of the distributed processes will perform and commit a transaction. If this is not enforced, the determination decisions for a particular transaction may be inconsistent. [c] In the Two-Phase Commit Protocol (2PC), why can blocking never be completely eliminated, even when the participants elect a new coordinator? After the election, the new coordinator may crash as well. In this case, the remaining participants may not be able to reach a final decision, because this requires the vote from the newly elected coordinator, just as before.

Hajautetut tietokannat Talvi 2016 Vastaukset Laskuharj. 5. CS-E4630 Distributed Databases Winter 2016 Answers Tutorial 5 9/9 [6] How do we compute the following SQL-clauses at dept. 10? Vihje: OR-osa voidaan yleensä korvata UNIONilla. Hint: the OR can usually be replaced by a UNION. At the headquarters (department nr 1) are stored not just the fragments for the employees but also replicas of all other fragments, meaning the whole relation is fully available at the headquarters. [i] select (*) from employee where department_number = 25. Lähetä kysely osastolle 25 (vaikkapa ilman where-ehtoa) ja lue sieltä vastaus. Send the query to dept. 25 (where clause not required) and read the answer from there. [ii] select avg(salary) from employee Kannattanee lähettää kokonaan osastolle 1. (Toinen vaihtoehto olisi kysyä kaikilta osastoilta, siis myös osastolta 1, d)-kohdan mukaisella tavalla.) Better to send the whole query to Dept 1 (can also ask depts, including dept. 1 as shown in case d below ). [iii] select department_number, avg(salary) from employee where department_number = 10 or department_number = 25 group by department_number Lähetä kysely Q1= Select 25, avg(salary) from employee osastolle 25 ja tee paikallisesti kysely Q2=select 10, avg(salary) from employee. Vastaus on union all näistä kahdesta: Q1 UNION ALL Q2 Send query Q1= Select 25, avg(salary) from employee to dept. 25 and make a local query Q2= Select 10, avg(salary) from employee. The answer is the union of these two: Q1 UNION ALL Q2. [iv] select avg(salary) from employee where department_number = 10 or department_number = 25. Lähetä kysely Q 25 = select count(*) as c, sum(salary) as s from employee osastolle 25 ja tee sama kysely paikallisesti (tulos Q10). Vastaus on select sum(s)/sum(c) from (Q10 union all Q 25 ). Tässä *voidaan* edelleen käyttää unionia ( OR-osaan ) ja olettaa, että tietokanta tietää, että avg-funktio voidaan laskea osissa summan ja lukumäärän avulla. Send query Q 25 = select count(*) as c, sum(salary) as s from employee to Dept. 25 and run same query locally at dept. 10, save as Q10. The answer is then given by: select sum(s)/sum(c) from (Q 10 union all Q 25 ). We *can* use the union here also (for the OR part) and break average into sum and count assuming the DB knows that the average can be computed in steps. [v] select avg(salary) from employee where department_number = 10 or department_number = 1. Kuten edellisessä kohdassa tai tässä kohdassa lienee edellistä kohtaa hyödyllisempää lähettää koko kysely osastolle 1, koska sen kanssa pitää kuitenkin kommunikoida. As in the previous case, or better yet, send whole query to Dept. 1, since have to communicate with it anyhow.