summaryrefslogtreecommitdiff
path: root/lib/libalpm/be_sync.c
Commit message (Collapse)AuthorAge
* Database read optimizationsDan McGee2011-08-28
| | | | | | | | | | | | | | | | | | | | | | Hard to believe there was still more room to improve on this, but I found an easily correctable oversight tonight. Our databases (both sync and local) contain many blank lines, and we were not moving onto the next line right away in these cases; instead we would proceed through our strcmp() conditional checks as normal. Some local numbers follow to show the effects of this patch: Sync `-Ss foobarbaz`: 71,709 blank lines skipped early ~1,505,889 strcmp() calls avoided (21 per line) ~15% speed improvement (.210 --> .179 sec) Local `-Qs foobarbaz`: 6,823 blank lines skipped early 115,991 strcmp() calls avoided (17 per line) ~6% speed improvement (.080 -> .071 sec) Signed-off-by: Dan McGee <dan@archlinux.org>
* be_sync.c: Fix memory leak in alpm_db_update()Lukas Fleischer2011-08-28
| | | | | | | Free "syncpath" and restore umask if we fail to grab a lock. Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de> Signed-off-by: Dan McGee <dan@archlinux.org>
* Remove argument from check_pgp_helperDan McGee2011-08-25
| | | | | | | This one wasn't all that necessary as we only used it in one place in the function, which can be checked easily enough at the call site. Signed-off-by: Dan McGee <dan@archlinux.org>
* Remove alpm_option_get_* usage from backendDan McGee2011-08-19
| | | | | | These are all available directly on the handle without indirection. Signed-off-by: Dan McGee <dan@archlinux.org>
* dload: add 'unlink_on_fail' to payload structDave Reisner2011-08-19
| | | | | | | | | Let callers of _alpm_download state whether we should delete on fail, rather than inferring it from context. We still override this decision and always unlink when a temp file is used. Signed-off-by: Dave Reisner <dreisner@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
* Enhance and utilize database status flagsDan McGee2011-08-18
| | | | | | | | | | | | | | | | | | * Move is_local standalone field to status enum * Create VALID/INVALID flag pair * Create EXISTS/MISSING flag pair With these additional fields, we can be more intelligent with database loading and messages to the user. We now only warn once if a sync database does not exist and do not continue to try to load it once we have marked it as missing. The reason for the flags existing in pairs is so the unknown case can be represented. There should never be a time when both flags in the same group are true, but if they are both false, it represents the unknown case. Care is taken to always manipulate both flags at the same time. Signed-off-by: Dan McGee <dan@archlinux.org>
* Only check necessary signatures and checksumsDan McGee2011-08-15
| | | | | | | | | | | The precedence goes as follows: signature > sha256sum > md5sum Add some logic and helper methods to check what we have available when loading a package, and then only check what is necessary to verify the package. This should speed up sync database verifies as we no longer will be doing both a checksum and a signature validation. Signed-off-by: Dan McGee <dan@archlinux.org>
* Parse conflicts/provides/replaces at database load timeDan McGee2011-08-15
| | | | | | | | | | | | | | | | | | | | | | We did this with depends way back in commit c244cfecf654d3 in 2007. We can do it with these fields as well. Of note is the inclusion of provides even though only '=' is supported- we'll parse other things, but no guarantees are given as to behavior, which is more or less similar to before since we only looked for the equals sign. Also of note is the non-inclusion of optdepends; this will likely be resolved down the road. The biggest benefactors of this change will be the resolving code that formerly had to parse and reparse several of these fields; it only happens once now at load time. This does lead to the disadvantage that we will now always be parsing this information up front even if we never need it in the split form, but as these are uncommon fields and our parser is quite efficient it shouldn't be a big concern. Signed-off-by: Dan McGee <dan@archlinux.org>
* Load and allow access to sha256sumDan McGee2011-08-15
| | | | | | | | | This adds a field in the package struct for this checksum type as well as allowing access via the API to it. The frontend is now able to display any read value. Note that this does not implement any use or verification of the value internally. Signed-off-by: Dan McGee <dan@archlinux.org>
* Don't trim whitespace when reading database entriesDan McGee2011-08-02
| | | | | | | | | | | We don't write with extra or unknown whitespace, so there is little reason for us to trim it when reading either. This also fixes the hopefully never encountered "paths that start or end with spaces" issue, for which two pactests have been added. The tests also contain other evil characters that we have encountered before and handle just fine, but it doesn't hurt to ensure we don't break such support in the future. Signed-off-by: Dan McGee <dan@archlinux.org>
* Clean up my debug logger messDan McGee2011-07-19
| | | | Signed-off-by: Dan McGee <dan@archlinux.org>
* Fix test suite when GPGME is disabledDan McGee2011-07-18
| | | | | | | | As noted by Allan, we failed pretty hard if gpgme was compiled out. With these changes, only sign001.py fails. This can/will be fixed later once we beef up the test suite with more signing tests anyway. Signed-off-by: Dan McGee <dan@archlinux.org>
* absorb some _alpm_download params into payload structDave Reisner2011-07-05
| | | | | | | Restore some sanity to the number of arguments passed to _alpm_download and curl_download_internal. Signed-off-by: Dave Reisner <dreisner@archlinux.org>
* lib/dload: prevent large file attacksDave Reisner2011-07-05
| | | | | | | | | | | | | | | | | | | This means creating a new struct which can pass more descriptive data from the back end sync functions to the downloader. In particular, we're interested in the download size read from the sync DB. When the remote server reports a size larger than this (via a content-length header), abort the transfer. In cases where the size is unknown, we set a hard upper limit of: * 25MiB for a sync DB * 16KiB for a signature For reference, 25MiB is more than twice the size of all of the current binary repos (with files) combined, and 16KiB is a truly gargantuan signature. Signed-off-by: Dave Reisner <dreisner@archlinux.org>
* dload: handle irregular URLsDave Reisner2011-07-05
| | | | | | | | | | | | | | | | | | | | | | URLs might end with a slash and follow redirects, or could be a generated by a script such as /getpkg.php?id=12345. In both cases, we may have a better filename that we can write to, taken from either content-disposition header, or the effective URL. Specific to the first case, we write to a temporary file of the format 'alpmtmp.XXXXXX', where XXXXXX is randomized by mkstemp(3). Since this is a randomly generated file, we cannot support resuming and the file is unlinked in the event of an interrupt. We also run into the possibility of changing out the filename from under alpm on a -U operation, so callers of _alpm_download can optionally pass a pointer to a *char to be filled in by curl_download_internal with the actual filename we wrote to. Any sync operation will pass a NULL pointer here, as we rely on specific names for packages from a mirror. Fixes FS#22645. Signed-off-by: Dave Reisner <d@falconindy.com>
* Allow invalid sync DBs to be returned by the libraryDan McGee2011-07-05
| | | | | | | | | | | | | | They are placeholders, but important for things like trying to re-sync a database missing a signature. By using the alpm_db_validity() method at the right time, a client can take the appropriate action with these invalid databases as necessary. In pacman's case, we disallow just about anything that involves looking at a sync database outside of an '-Sy' operation (although we do check the validity immediately after). A few operations are still permitted- '-Q' ops that don't touch sync databases as well as '-R'. Signed-off-by: Dan McGee <dan@archlinux.org>
* signing: move to new signing verification and return schemeDan McGee2011-07-05
| | | | | | | | | | | | | | | | | This gives us more granularity than the former Never/Optional/Always trifecta. The frontend still uses these values temporarily but that will be changed in a future patch. * Use 'siglevel' consistenly in method names, 'level' as variable name * The level becomes an enum bitmask value for flexibility * Signature check methods now return a array of status codes rather than a simple integer success/failure value. This allows callers to determine whether things such as an unknown signature are valid. * Specific signature error codes mostly disappear in favor of the above returned status code; pm_errno is now set only to PKG_INVALID_SIG or DB_INVALID_SIG as appropriate. Signed-off-by: Dan McGee <dan@archlinux.org>
* Merge remote-tracking branch 'allan/ALPM'Dan McGee2011-07-03
|\ | | | | | | | | | | | | | | | | | | Conflicts: lib/libalpm/be_local.c lib/libalpm/be_package.c lib/libalpm/conflict.c lib/libalpm/diskspace.c lib/libalpm/dload.c lib/libalpm/remove.c
| * Prefix _alpm_errno_t members with ALPMAllan McRae2011-07-02
| | | | | | | | Signed-off-by: Allan McRae <allan@archlinux.org>
| * Prefix alpm_loglevel_t members with ALPMAllan McRae2011-07-02
| | | | | | | | Signed-off-by: Allan McRae <allan@archlinux.org>
* | be_sync: make READ_NEXT() a no-arg macroDan McGee2011-07-03
|/ | | | | | | | We passed in 'line', but not 'buf.line'. In addition, the macros building off of READ_NEXT() assume variable names anyway. Since we only use these macros in one function, might as well simplify them. Signed-off-by: Dan McGee <dan@archlinux.org>
* Make alpm_db_get_sigverify_level() publicDan McGee2011-06-30
| | | | Signed-off-by: Dan McGee <dan@archlinux.org>
* Add an alpm_db_get_valid() public functionDan McGee2011-06-30
| | | | | | This allows one to check if a database is valid or invalid. Signed-off-by: Dan McGee <dan@archlinux.org>
* Rename pmpkg_t to alpm_pkg_tAllan McRae2011-06-28
| | | | Signed-off-by: Allan McRae <allan@archlinux.org>
* Rename pmdb_t to alpm_db_tAllan McRae2011-06-28
| | | | Signed-off-by: Allan McRae <allan@archlinux.org>
* Rename pmhandle_t to alpm_handle_tAllan McRae2011-06-28
| | | | Signed-off-by: Allan McRae <allan@archlinux.org>
* Make sync DB reading a bit more flexibleDan McGee2011-06-24
| | | | | | | | | | | We can reorganize things a bit to not require reading a directory-only entry first (or at all). This was noticed while working on some pactest improvements, but should be a good step forward anyway. Also make _alpm_splitname() a bit more generic in where it stores the data it parses. Signed-off-by: Dan McGee <dan@archlinux.org>
* Don't require a transaction for sync DB updatesDan McGee2011-06-24
| | | | | | | Instead, just do the required locking directly in the backend in calls to alpm_db_update(). Signed-off-by: Dan McGee <dan@archlinux.org>
* Add a 'valid' flag to the database objectDan McGee2011-06-24
| | | | | | | | | | | | | | Start by converting all of our flags to a 'status' bitmask (pkgcache status, grpcache status). Add a new 'valid' flag as well. This will let us keep track if the database itself has been marked valid in whatever fashion. For local databases at the moment we ensure there are no depends files; for sync databases we ensure the PGP signature is valid if required/requested. The loading of the pkgcache is prohibited if the database is invalid. Signed-off-by: Dan McGee <dan@archlinux.org>
* Move database 'version' check to registration timeDan McGee2011-06-24
| | | | | | | | This is another step toward doing both local database validation (ensuring we don't have depends files) and sync database validation (via signatures if present) when the database is registered. Signed-off-by: Dan McGee <dan@archlinux.org>
* Do database signature checking at load timeDan McGee2011-06-24
| | | | | | | | | | | | | | | | | | This is the ideal place to do it as all clients should be checking the return value and ensuring there are no errors. This is similar to pkg_load(). We also add an additional step of validation after we download a new database; a subsequent '-y' operation can potentially invalidate the original check at registration time. Note that this implementation is still a bit naive; if a signature is invalid it is currently impossible to refresh and re-download the file without manually deleting it first. Similarly, if one downloads a database and the check fails, the database object is still there and can be used. These shortcomings will be addressed in a future commit. Signed-off-by: Dan McGee <dan@archlinux.org>
* Prevent segfault when parsing unexpected sync database fileDan McGee2011-06-24
| | | | | | | | | This doesn't fix the real (bigger) problem of failing to parse sync databases without directory entries, but it does prevent the parser from segfaulting when the first desc file encountered did not have a directory entry, among other conditions. Signed-off-by: Dan McGee <dan@archlinux.org>
* Documented purpose of be_*.cKerrick Staley2011-06-20
| | | | | | | | Added a line to the top of each of be_local.c, be_package.c, and be_sync.c indicating their purposes. Signed-off-by: Kerrick Staley <mail@kerrickstaley.com> Signed-off-by: Dan McGee <dan@archlinux.org>
* Ensure handle is valid and pm_errno is reset when calling into APIDan McGee2011-06-14
| | | | | | | | | | | | | | | | | We didn't do due diligence before and ensure prior pm_errno values weren't influencing what happened in further ALPM calls. I observed one case of early setup code setting pm_errno to PM_ERR_WRONG_ARGS and that flag persisting the entire time we were calling library code. Add a new CHECK_HANDLE() macro that does two things: 1) ensures the handle variable passed to it is non-NULL and 2) clears any existing pm_errno flag set on the handle. This macro can replace many places we used the ASSERT(handle != NULL, ...) pattern before. Several other other places only need a simple 'set to zero' of the pm_errno field. Signed-off-by: Dan McGee <dan@archlinux.org>
* alpm_db_update(): refactor out sync dir create/checkDan McGee2011-06-13
| | | | | | This was a lot of stuff that can stand by itself for the most part. Signed-off-by: Dan McGee <dan@archlinux.org>
* Switch all logging to use handle directlyDan McGee2011-06-13
| | | | | | | This is the last user of our global handle object. Once again the diff is large but the functional changes are not. Signed-off-by: Dan McGee <dan@archlinux.org>
* Move pm_errno onto the handleDan McGee2011-06-13
| | | | | | This involves some serious changes and a very messy diff, unfortunately. Signed-off-by: Dan McGee <dan@archlinux.org>
* Require handle argument to all alpm_option_(get|set)_*() methodsDan McGee2011-06-09
| | | | | | | | This requires a lot of line changes, but not many functional changes as more often than not our handle variable is already available in some fashion. Signed-off-by: Dan McGee <dan@archlinux.org>
* Plug a memory leakDan McGee2011-06-08
| | | | | | Introduced by me in commit cc25576f8b54b3. Signed-off-by: Dan McGee <dan@archlinux.org>
* Use standard errno codes in return from _alpm_archive_fgetsDan McGee2011-06-03
| | | | | | | | | | | | | This allows us to not require the context (e.g. handle) when calling this function. Also beef up the checks in the two callers of this function to bail if the last return code is not ARCHIVE_EOF, which is the expected value. This requires a change to one of the pactest return codes and the overall result of the test, but results in a much safer operating condition whereby invalid database entries will stop the operation. Signed-off-by: Dan McGee <dan@archlinux.org>
* Remove global handle from some package and db codeDan McGee2011-06-03
| | | | Signed-off-by: Dan McGee <dan@archlinux.org>
* Push down extern handle variable to files that need itDan McGee2011-06-03
| | | | | | | This will make the patching process less invasive as we start to remove this variable from all source files. Signed-off-by: Dan McGee <dan@archlinux.org>
* Add handle attribute to pmpkg_t structDan McGee2011-06-03
| | | | | | | Similar to what we just did for the database; this will make it easy to always know what handle a given package originated from. Signed-off-by: Dan McGee <dan@archlinux.org>
* Add handle attribute to pmdb_t structDan McGee2011-06-03
| | | | | | | | This is the first step in a long process to remove our dependence on the global handle variable we currently share in libalpm, with the goal to make things a bit more thread-safe and re-entrant. Signed-off-by: Dan McGee <dan@archlinux.org>
* Remove ALPM_LOG_FUNC macroDan McGee2011-06-03
| | | | | | | | | The usefulness of this is rather limited due to it not being compiled into production builds. When you do choose to see the output, it is often overwhelming and not helpful. The best bet is to use a debugger and/or well-placed fprintf() statements. Signed-off-by: Dan McGee <dan@archlinux.org>
* Remove sync DB reregister checkDan McGee2011-05-05
| | | | | | | It's your own damn fault if you do this, and this code is remnants from an old time when we weren't very good at coding. Signed-off-by: Dan McGee <dan@archlinux.org>
* Merge branch 'maint'Dan McGee2011-05-05
|\
| * Ensure populate error return codes are consistentDan McGee2011-05-05
| | | | | | | | | | | | | | It must be -1 to differentiate it from a number of packages loaded count. Signed-off-by: Dan McGee <dan@archlinux.org>
* | Logging changes during DB loadDan McGee2011-05-05
| | | | | | | | | | | | | | | | The switch from FUNCTION to DEBUG was ill-advised inside the local database load. Instead, add a DEBUG level logger to both local and sync database loads that shows the number of packages processed. Signed-off-by: Dan McGee <dan@archlinux.org>
* | Deal with unused function parameters correctlyAllan McRae2011-05-04
| | | | | | | | | | | | | | | | | | | | | | This started off removing the "(void)foo" hacks to work around unused function parameters and ended up fixing every warning generated by -Wunused-parameter. Dan: rename to UNUSED. Signed-off-by: Allan McRae <allan@archlinux.org> Signed-off-by: Dan McGee <dan@archlinux.org>
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-05 10:55:21 +0000