summaryrefslogtreecommitdiff
path: root/lib/libalpm/be_files.c
Commit message (Collapse)AuthorAge
* Complete rework of package accessor logicDan McGee2010-10-13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Hopefully we've finally arrived at package handling nirvana, or at least this commit will get us a heck of a lot closer. The former method of getting the depends list for a package was the following: 1. call alpm_pkg_get_depends() 2. this method would check if the package came from the cache 3. if so, ensure our cache level is correct, otherwise call db_load 4. finally return the depends list Why did this suck? Because getting the depends list from the package shouldn't care about whether the package was loaded from a file, from the 'package cache', or some other system which we can't even use because the damn thing is so complicated. It should just return the depends list. So what does this commit change? It adds a pointer to a struct of function pointers to every package for all of these 'package operations' as I've decided to call them (I know, sounds completely straightforward, right?). So now when we call an alpm_pkg_get-* function, we don't do any of the cache logic or anything else there- we let the actual backend handle it by delegating all work to the method at pkg->ops->get_depends. Now that be_package has achieved equal status with be_files, we can treat packages from these completely different load points differently. We know a package loaded from a zip file will have all of its fields populated, so we can set up all its accessor functions to be direct accessors. On the other hand, the packages loaded from the local and sync DBs are not always fully-loaded, so their accessor functions are routed through the same logic as before. Net result? More code. However, this code now make it roughly 52 times easier to open the door to something like a read-only tar.gz database backend. Are you still reading? I'm impressed. Looking at the patch will probably be clearer than this long-winded explanation. Signed-off-by: Dan McGee <dan@archlinux.org> [Allan: rebase and adjust] Signed-off-by: Allan McRae <allan@archlinux.org>
* Allow local and sync db to be treated separatelyDan McGee2010-10-13
| | | | | | | | | | | | | | | | | | | | | | | Implement this seemingly simple change in package.h: typedef enum _pmpkgfrom_t { - PKG_FROM_CACHE = 1, - PKG_FROM_FILE + PKG_FROM_FILE = 1, + PKG_FROM_LOCALDB, + PKG_FROM_SYNCDB } pmpkgfrom_t; which requires flushing out several assumptions from around the codebase with regards to usage of the PKG_FROM_CACHE value. Make some changes where required to allow the switch, and now the correct value should be set (via a crude hack) depending on whether a package was loaded as an entry in a local db or a sync db. This patch underwent some big rebasing from Allan and Dan. Signed-off-by: Dan McGee <dan@archlinux.org> Signed-off-by: Allan McRae <allan@archlinux.org>
* Avoid stat call to determine is_directory if possibleDan McGee2010-10-11
| | | | | | | | | | On Linux and OS X, we can determine if an entry obtained through a readdir() call is a directory without also having to stat it. This can save a significant number of syscalls. The performance increase isn't dramatic, but it could be on some platforms (e.g. Cygwin) so it shouldn't hurt to use this unconditionally where supported. Signed-off-by: Dan McGee <dan@archlinux.org>
* fgets invocation cleanupDan McGee2010-08-23
| | | | | | | | | | | | | | From the fgets manpage: fgets() reads in at most one less than size characters from stream and stores them into the buffer pointed to by s. Reading stops after an EOF or a newline. If a newline is read, it is stored into the buffer. A '\0' is stored after the last character in the buffer. This means there is no need at all to do 'size - 1' math. Remove all of that and just use sizeof() for simplicity on the buffer we plan on reading into. Signed-off-by: Dan McGee <dan@archlinux.org>
* Download sync db into DBPath/syncAllan McRae2010-07-07
| | | | | | | | | | | The sync db should be stored in the sync/ folder. This cleans up DBPath to only have local/ and sync/ directories in it. A nice side effect is that the db are now in the right place so we can implement directly reading from them. Signed-off-by: Allan McRae <allan@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
* Remove DBEXT usageAllan McRae2010-07-07
| | | | | | | | | With commit 5dffef78, the repo database always has a symlink of the form reponame.db. Use that filename and let libarchive determine the compression type. Signed-off-by: Allan McRae <allan@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
* Check return value of fgets callsAllan McRae2010-07-07
| | | | | | | Prevents compiler warnings with -D_FORTIFY_SOURCE=2 Signed-off-by: Allan McRae <allan@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
* More consistent printing of off_t and time_tXavier Chantry2010-03-25
| | | | | | | | time_t : %ld off_t : %jd and cast to intmax_t Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
* Unbreak the database partial extraction codeDan McGee2010-03-15
| | | | | | | | | | | | | Basically I'm the idiot that thought I could make it better and completely forgot how freeing the contents of the original lists would screw up our nice little diff extraction lists. This caused segfaults among other problems. Last time I try to do that... Program received signal SIGSEGV, Segmentation fault. 0x00007ffff627ce26 in strcmp () from /lib/libc.so.6 (gdb) bt Signed-off-by: Dan McGee <dan@archlinux.org>
* fix a few warnings reported by clangXavier Chantry2010-03-14
| | | | | | | | | - remove unused variables - some more sanity checks - safer printf Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
* Reduce duplicate code in DB extractionDan McGee2010-03-14
| | | | | | | | Follow-up to the previous "Only extract new DB entries" patch; move the partial extraction code inside one side of the loop so we can use the same code for actually doing file extraction. Signed-off-by: Dan McGee <dan@archlinux.org>
* Only extract new DB entriesXavier Chantry2010-03-14
| | | | | | | | | | | | | | | | | | | | | | | | This implements FS#15198. The idea apparently came from Csaba Henk <csaba-ml <at> creo.hu> which submitted a patch to Frugalware, so thanks to him, even though I did not look at the code :) The idea is to only extract folders for new packages into the package database and clean up the old directories. This is essentially implementing Xyne's "rebase" script within pacman. If using -Syy, just remove and extract everything. If using -Sy : 1. Generate list of directories in DB 2. Generate list of directories in archive 3. Compare both 4. Clean up old directories 5. Extract new directories Original-work-by: Allan McRae <allan@archlinux.org> Signed-off-by: Xavier Chantry <shiningxc@gmail.com> [Dan: fix compile error, s/int/size_t/] Signed-off-by: Dan McGee <dan@archlinux.org>
* Print error on duplicated database entriesNagy Gabor2009-12-13
| | | | | | | | | | | | Some users reported duplicated database entries in /var/lib/pacman/local/, for example, both foo-1.0-1 and foo-2.0-1 subdirectories existed. (Bogus 3rd-party scripts, backup?) In this case pacman reported no error and its behaviour was mysterious. From now on, pacman detects this situation and prints an error message. Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu> Signed-off-by: Dan McGee <dan@archlinux.org>
* download: major refactor to address lingering issuesDan McGee2009-11-15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Sorry for this being such a huge patch, but I believe it is necessary for quite a few reasons which I will attempt to explain herein. I've been mulling this over for a while, but wasn't super happy with making the download interface more complex. Instead, if we carefully order things in the internal download code, we can actually make the interface simpler. 1. FS#15657 - This involves `name.db.tar.gz.part` files being left around the filesystem, and then causing all sorts of issues when someone attempts to rerun the operation they canceled. We need to ensure that if we resume a download, we are resuming it on exactly the same file; if we cannot be almost postive of that then we need to start over. 2. http://www.mail-archive.com/pacman-dev@archlinux.org/msg03536.html - Here we have a lighttpd bug to ruin the day. If we send both a Range: header and If-Modified-Since: header across the wire in a GET request, lighttpd doesn't do what we want in several cases. If the file hadn't been modified, it returns a '304 Not Modified' instead of a '206 Partial Content'. We need to do a stat (e.g. HEAD in HTTP terms) operation here, and the proceed accordingly based off the values we get back from it. 3. The mtime stuff was rather ugly, and relied on the called function to write back to a passed in reference, which isn't the greatest. Instead, use the power of the filesystem to contain this info. Every file downloaded internally is now carefully timestamped with the remote file time. This should allow the resume logic to work. In order to guarantee this, we need to implement a signal handler that catches interrupts, notifies the running code, and causes it to set the mtimes on the file. It then rethrows the signal so the pacman signal handler (or any frontend) works as expected. 4. We did a lot of funky stuff in trying to track the DB last modified time. It is a lot easier to just keep the downloaded DB file around and track the time on that rather than in a funky dot file. It also kills a lot of code. 5. For GPG verification of the databases down the road, we are going to need the DB file around for at least a short bit of time anyway, so this gets us closer to that. Signed-off-by: Dan McGee <dan@archlinux.org> [Xav: fixed printf with off_t] Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
* Size handling was changed in fgets() functionsLaszlo Papp2009-10-19
| | | | | | | | Pacman's fgets function in the API used hardcoded numbers to identify the size. This is not good practice, so replace them with sizeof handling. Signed-off-by: Laszlo Papp <djszapi@archlinux.us> Signed-off-by: Dan McGee <dan@archlinux.org>
* Rework the alpm_unpack functionsXavier Chantry2009-10-11
| | | | | | | Add support to extract a list of entries Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
* cygwin fix : use unsigned char for ctype functionXavier Chantry2009-10-11
| | | | | | | | | | | | See http://www.nabble.com/-PATCH-RFA--Distinguish-between-EOF-and-character-with-value-0xff-td23161772.html#a23188494 cygwin 1.7 actually displays a warning when using signed char with the ctype function, so that compilation fails when using -Wall -Werror. So we just cast all arguments to unsigned char. Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
* Fully implement database lazy loadingDan McGee2009-09-20
| | | | | | | | | | | | | | | Commit 34e1413d75 attempted to implement lazy loading of package databases. Although it took care of my main complaint (creating the database directory if it didn't exist), it didn't allow sync repos to be registered before alpm_option_set_dbpath() had been called. With this patch, we no longer compute the individual repository DB paths until necessary, allowing full lazy loading to work as intended, and allowing us to drop the extra setlibpath() calls from the frontend. This allows the changes introduced in a2cd48960 (but later reverted) to be added back in again. Signed-off-by: Dan McGee <dan@archlinux.org>
* Merge branch 'maint'Dan McGee2009-09-20
|\ | | | | | | | | | | | | | | Message updates made this one a bit messy, but nothing too bad. Conflicts: lib/libalpm/add.c lib/libalpm/remove.c
| * String improvementsXavier Chantry2009-09-16
| | | | | | | | | | | | | | Add more untranslated strings, improve consistency, etc. Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
* | Remove transaction typeXavier Chantry2009-09-08
|/ | | | | | | | | | | | | | | This basically started with this change : /* Transaction */ struct __pmtrans_t { - pmtranstype_t type; pmtransflag_t flags; pmtransstate_t state; - alpm_list_t *packages; /* list of (pmpkg_t *) */ + alpm_list_t *add; /* list of (pmpkg_t *) */ + alpm_list_t *remove; /* list of (pmpkg_t *) */ And then I have to modify all the code accordingly.
* Fix non-C style comments in libalpmGerardo Exequiel Pozzi2009-05-11
| | | | | Signed-off-by: Gerardo Exequiel Pozzi <vmlinuz386@yahoo.com.ar> Signed-off-by: Dan McGee <dan@archlinux.org>
* Add detail documentation for alpm_db_update()Sebastian Nowicki2009-04-11
| | | | | | | This adds alpm_db_update() to the alpm_databases Doxygen group. The function is described in more detail and a code example is given. Signed-off-by: Dan McGee <dan@archlinux.org>
* be_files.c : use %jd instead of %ju for intmax_tXavier Chantry2009-03-15
| | | | | Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
* Delay the creation of local and sync db dir.Xavier Chantry2009-01-20
| | | | | | | | | | | | | | We don't need to create the directories when local or sync dbs are registered. For example, if a sync db does not exist, we cannot even do "pacman -Q" as an user. Instead, we can create the local db if needed during the db_prepare operation, and sync dbs on db_update. Also remove some more useless abstractions in db_update and switch to a much more efficient way to remove a sync db : rm -rf. Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
* Remove some db abstraction crap.Xavier Chantry2009-01-20
| | | | | | | | These db_open and db_close looked quite useless. And they caused the db directory to be opened on a simple registering of a database. This is totally unneeded, this opening can be delayed to when we actually need it. Signed-off-by: Xavier Chantry <shiningxc@gmail.com>
* Rename alpm_get_md5sum to alpm_compute_md5sum and alpm_dep_get_string to ↵Nagy Gabor2009-01-12
| | | | | | | | | | | alpm_dep_compute_string This patch introduces the following function name convention: _compute_ in function name: the return value must be freed. _get_ in function name: the return value must not be freed. Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu> Signed-off-by: Dan McGee <dan@archlinux.org>
* Separate local db directory creation and db writeAllan McRae2009-01-02
| | | | | | | | | | | | | | | Changelogs and install files were getting extracted into the local db folder before it was manually created. This created issues for uses with 0077 umasks and was highlighted with the new sudo handling of umasks (FS#12263). This moves the local db creation to its own function which is called before the start of package archive extraction. Also, added a check that the folder is actually created. Signed-off-by: Allan McRae <allan@archlinux.org> [Dan: rename to _alpm_db_prepare()] Signed-off-by: Dan McGee <dan@archlinux.org>
* Add regex to delta code so we don't segfault when reading lineDan McGee2008-11-30
| | | | | | | If the delta line doesn't match our regex, we won't go and process it, possibly walking off the end of the string. Signed-off-by: Dan McGee <dan@archlinux.org>
* Minor code cleanupsDan McGee2008-11-17
| | | | | | Mostly noticed when compiling libalpm/pacman with ICC. Signed-off-by: Dan McGee <dan@archlinux.org>
* Use access() instead of stat() when possibleDan McGee2008-06-15
| | | | | | | | | | | | | | | | | | | We were using the stat() system call in quite a few places when we didn't actually need anything the stat struct returned- we were simply checking for file existence. access() will be more efficient in those cases. Before (strace pacman -Ss pacman): % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 33.16 0.005987 0 19016 stat64 After: % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 34.85 0.003863 0 12633 1 access 7.95 0.000881 0 6391 7 stat64 Signed-off-by: Dan McGee <dan@archlinux.org>
* Get rid of double / in database paths.Xavier Chantry2008-06-04
| | | | | | | | | | | | | | | | | | | Errors like the following one happen regularly (for unknown reasons...) : error: could not open file /var/lib/pacman/local//glibc-2.7-9/depends: No such file or directory Anyway, every time an user reported an error like that, it always seemed like he thought the error was caused by the double /, which is obviously wrong. Since db->path always include a trailing /, there is no need to add one when concatenating paths in be_files.c or add.c. Additionally, some static strings were switched to dynamic. And the computation of the "dbpath"/"pkgname"-"pkgversion" was refactored in db_read, db_write and db_remove with a get_pkgpath static function. Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
* Use correct C type for file sizesDan McGee2008-06-04
| | | | | | | | | | | | | | We have been using unsigned long as a file size type for a while, which works but isn't quite correct and could easily break. Worse was probably our use of int in the download callback functions, which could be restrictive for packages > 2GB in size. Switch all file size variables to use off_t, which is the preferred type for file sizes. Note that at least on Linux, all applications compiled against libalpm must now be sure to use large file support, where _FILE_OFFSET_BITS is defined to be 64 or there will be some weird issues that crop up. Signed-off-by: Dan McGee <dan@archlinux.org>
* be_files : stores REPLACES and FORCE in desc.Xavier Chantry2008-05-29
| | | | | | | | | | | | | | | repo-add and db_read both assume that REPLACES and FORCE fields are in the desc file, so do that for db_write as well (instead of depends file). Note that db_write is currently only used on the local database. And the only purpose of replaces and force in local database is for information purpose (available on -Qi operations). So this is not a big problem. Ref: http://www.archlinux.org/pipermail/pacman-dev/2008-May/011859.html Acked-by: Dan McGee <dan@archlinux.org> Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
* Add some verbosity to _alpm_db_read.Nagy Gabor2008-05-29
| | | | | | | | | | | Name and version are computed from "/var/lib/pacman/..." pathname. And the %NAME% and %VERSION% fields from the desc file were not even read. So now, when we read the desc file, we make sure the %NAME% and %VERSION% fields are consistent. Signed-off-by: Nagy Gabor <ngaba@bibl.u-szeged.hu> Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
* Simplify _alpm_pkg_new()Dan McGee2008-05-13
| | | | | | | | Any real call of this function doesn't specify a name or version ahead of time, so just kill that functionality off. Now to remove those dummy packages... Signed-off-by: Dan McGee <dan@archlinux.org>
* Fix rewinddir regression by cleaning up db_scanXavier Chantry2008-05-13
| | | | | | | | | | | | | | | | | | Commit 046003844739416ff6d168dd2dec76490adb0727 caused a regression when rereading the pkgcache after updating the on-disk databases. A rewinddir call was errantly removed. Instead of replacing the call to rewindir, clean up this whole mess. db_scan is used only once and with target == NULL so there was actually half the code of db_scan which was unused. This is gone now and replaced by a single new db_populate function. Dan: add_sorted ended up being 3x slower than one msort at the end, so I changed back to that. I also made one pointer variable const and merged this whole patch with my original fix for the rewinddir issue. Signed-off-by: Xavier Chantry <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
* Remove some useless abstraction and start db cleanupDan McGee2008-05-11
| | | | | | | | | | We have some useless abstractions like an alpm_db_rewind function. I've read somewhere that readdir() was the worst filesystem function call invented, and what do we do? Add a wrapper around it. Kill this abstraction and move some other things into be_files that should be there anyway because they are so tied to how a files backend works. Signed-off-by: Dan McGee <dan@archlinux.org>
* be_files.c: PATH_MAX cleanupDan McGee2008-05-08
| | | | | | | Most of these are not easy to remove, but I could kill the ones in the two lastupdate functions. Signed-off-by: Dan McGee <dan@archlinux.org>
* Remove unnecessary header file, move one macro to util.cDan McGee2008-04-06
| | | | Signed-off-by: Dan McGee <dan@archlinux.org>
* Move the deptest code from frontend to backend.Chantry Xavier2008-01-21
| | | | | | | | | | | | | | | | | | The deptest code (pacman -T) used by makepkg was mostly in the frontend. There were 2 drawbacks: 1) the public splitdep function returns a pmdepend_t struct, but the _alpm_dep_free function for freeing it is private. So there was a memleak. 2) there is a helper in the backend (satisfycmp in deps.c) which makes this function much easier. So this adds a new public alpm_deptest in libalpm/deps.c, which cleans pacman_deptest in pacman/deptest.c a lot. Besides, alpm_splitdep was made private, because the frontend no longer requires it, and _alpm_dep_free is also private. Finally the deptest001 pactest was extended. Signed-off-by: Chantry Xavier <shiningxc@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
* Convert deltas to dynamic allocationDan McGee2008-01-21
| | | | | | | | Another elimination of a static length structure in libalpm. Should result in a little more memory saved during execution of packages with lots of deltas attached. Signed-off-by: Dan McGee <dan@archlinux.org>
* Store replaces in the local DB tooAaron Griffin2008-01-12
| | | | | | Signed-off-by: Aaron Griffin <aaronmgriffin@gmail.com> [Dan: might as well store 'force' too] Signed-off-by: Dan McGee <dan@archlinux.org>
* Use dynamic string allocation in package structuresDan McGee2008-01-11
| | | | | | | | | | | | | | This also affects all structures with static strings, such as depmiss, conflict, etc. This should help a lot with memory usage, and hopefully make things a bit more "idiot proof". Currently our pactest pass/fail rate is identical before and after this patch. This is not to say it is a perfect patch- I have yet to pull valgrind out. However, this should be quite safe to use in all situations from here on out, and we can start plugging the memleaks. Original-work-by: Aaron Griffin <aaronmgriffin@gmail.com> Signed-off-by: Dan McGee <dan@archlinux.org>
* Remove gettext calls from debug-level messagesDan McGee2008-01-01
| | | | | | | | | | These used FUNCTION output level and not DEBUG, so I didn't catch them way back when I removed those gettext calls. Remove them now (which exposed a nice little memory access error elsewhere in the code). This should have a slight speedup effect on the code too as we no longer have to make the gettext call even when these messages aren't printed. Signed-off-by: Dan McGee <dan@archlinux.org>
* Update GNU GPL boilerplate and copyright datesDan McGee2007-12-10
| | | | | | | Update the GPL boilerplate to direct people to the GNU website for a copy of the license, as well as bump all of Judd's copyrights to 2007. Signed-off-by: Dan McGee <dan@archlinux.org>
* Add missing header includes for setlocaleDan McGee2007-11-20
| | | | Signed-off-by: Dan McGee <dan@archlinux.org>
* War on whitespaceDan McGee2007-11-16
| | | | | | Run the kernel's cleanfile script on all of our source files. Signed-off-by: Dan McGee <dan@archlinux.org>
* libalpm: simplify sync db lastupdateDan McGee2007-11-16
| | | | | | | | | | | Legacy code is hitting the trash here. Remove unnecessary _alpm_time2string time storage abstraction in favor of just writing the time_t value to the disk. The only drawback is that everyone's sync DBs will have to be updated at least once so that the lastupdate values are stored right. :) Signed-off-by: Dan McGee <dan@archlinux.org>
* Move alpm_splitdep usage to db_readDan McGee2007-11-14
| | | | | | | | | | | | | Holy inefficient batman! For a pacman -Qt operation (when we are using compute_requiredby and not database entries), splitdep was being called ~1.3 million times on my local database. By splitting when we read the DB, we drop this number to around 1700 and save a LOT of time in doing so (a 5x increase in pacman -Qt speed here). Note that the depends alpm_list_t in the package struct is no longer a string list, but a list of pmdepent_t objects. Signed-off-by: Dan McGee <dan@archlinux.org>
generated by cgit v1.2.3-70-g09d2 (git 2.47.0) at 2024-11-19 07:36:23 +0000