Atmosphere/libraries/libstratosphere/source/fssystem/fssystem_directory_savedata_filesystem.cpp
Adubbz c7026b9094
Implement the NCM sysmodule (closes #91)
* Implement NCM

* Modernize ncm_main

* Remove unnecessary smExit

* Give access to svcCallSecureMonitor

* Stack size bump

* Fix incorrect setup for NandUser's content storage entry

* Fix a potential data abort when flushing the placeholder accessor cache

* Fix HasFile and HasDirectory

* Use r+b, not w+b

* Misc fixes

* errno begone

* Fixed more stdio error handling

* More main fixes

* Various command improvements

* Make dispatch tables great again

* Fix logic inversion

* Fixed content path generation

* Bump heap size, fix CleanupAllPlaceHolder

* Various fixes. Note: This contains debug stuff which will be removed later. I was getting tired of having to cherrypick tiny changes

* Fixed placeholder/content deletion

* Fixed incorrect content manager destruction

* Prevent automatic placeholder creation on open

* Fixed List implementation. Also lots of debug logging.

* Removed debug code

* Added a scope guard for WritePlaceHolder

* Manually prevent placeholder/content appending

* Revert "Removed debug code"

This reverts commit d6ff261fcc.

* Always cache placeholder file. Switch to ftell for preventing appending

* Universally use EnsureEnabled

* Abstract away file writing logic

* Misc cleanup

* Refactor placeholder cacheing

* Remove debug code (again)

* Revert "Remove debug code (again)"

This reverts commit 168447d80e.

* Misc changes

* Fixed file modes

* Fixed ContentId/PlaceHolderId alignment

* Improved type safety

* Fixed reinitialization

* Fixed doubleup on path creation

* Remove debug code

* Fixed 1.0.0 booting

* Correct amount of add on content

* Correct main thread stack size

* lr: Introducing registered data

* Reorder stratosphere Makefile

* Move results to libstrat

* lr: Cleanup lr_redirection

* lr: lr_manager tweaks

* lr: Imrpoved path handling and adjust ResolveAddOnContentPath order

* lr: Organise types

* Add eof newlines

* lr: Eliminate unnecessary vars

* lr: Unnecessary vars 2 electric boogaloo

* lr: Various helpers

* lr: RegisteredLocationResolver helpers

* ncm: Move ncm_types to libstrat

* ncm: Misc cleanup

* Implement NCM

* Modernize ncm_main

* Remove unnecessary smExit

* Give access to svcCallSecureMonitor

* Stack size bump

* Fix incorrect setup for NandUser's content storage entry

* Fix a potential data abort when flushing the placeholder accessor cache

* Fix HasFile and HasDirectory

* Use r+b, not w+b

* Misc fixes

* errno begone

* Fixed more stdio error handling

* More main fixes

* Various command improvements

* Make dispatch tables great again

* Fix logic inversion

* Fixed content path generation

* Bump heap size, fix CleanupAllPlaceHolder

* Various fixes. Note: This contains debug stuff which will be removed later. I was getting tired of having to cherrypick tiny changes

* Fixed placeholder/content deletion

* Fixed incorrect content manager destruction

* Prevent automatic placeholder creation on open

* Fixed List implementation. Also lots of debug logging.

* Removed debug code

* Added a scope guard for WritePlaceHolder

* Manually prevent placeholder/content appending

* Revert "Removed debug code"

This reverts commit d6ff261fcc.

* Always cache placeholder file. Switch to ftell for preventing appending

* Universally use EnsureEnabled

* Abstract away file writing logic

* Misc cleanup

* Refactor placeholder cacheing

* Remove debug code (again)

* Revert "Remove debug code (again)"

This reverts commit 168447d80e.

* Misc changes

* Fixed file modes

* Fixed ContentId/PlaceHolderId alignment

* Improved type safety

* Fixed reinitialization

* Fixed doubleup on path creation

* Remove debug code

* Fixed 1.0.0 booting

* Correct amount of add on content

* Correct main thread stack size

* lr: Introducing registered data

* Reorder stratosphere Makefile

* Move results to libstrat

* lr: Cleanup lr_redirection

* lr: lr_manager tweaks

* lr: Imrpoved path handling and adjust ResolveAddOnContentPath order

* lr: Organise types

* Add eof newlines

* lr: Eliminate unnecessary vars

* lr: Unnecessary vars 2 electric boogaloo

* lr: Various helpers

* lr: RegisteredLocationResolver helpers

* ncm: Move ncm_types to libstrat

* ncm: Misc cleanup

* Updated AddOnContentLocationResolver and RegisteredLocationResolver to 9.0.0

* Finished updating lr to 9.0.0

* Updated NCM to 9.0.0

* Fix libstrat includes

* Fixed application launching

* title_id_2 -> owner_tid

* Updated to new-ipc

* Change to using pure virtuals

* Title Id -> Program Id

* Fixed compilation against master

* std::scoped_lock<> -> std::scoped_lock

* Adopted R_UNLESS and R_CONVERT

* Prefix namespace to Results

* Adopt std::numeric_limits

* Fixed incorrect error handling in ReadFile

* Adopted AMS_ABORT_UNLESS

* Adopt util::GenerateUuid()

* Syntax improvements

* ncm_types: Address review

* Address more review comments

* Updated copyrights

* Address more feedback

* More feedback addressed

* More changes

* Move dispatch tables out of interface files

* Addressed remaining comments

* lr: move into libstratosphere

* ncm: Fix logic inversion

* lr: Add comments

* lr: Remove whitespace

* ncm: Start addressing feedback

* ncm: Cleanup InitializeContentManager

* lr: support client-side usage

* lr_service -> lr_api

* ncm: Begin refactoring content manager

* ncm: More content manager improvements

* ncm: Content manager mount improvements

* ldr: use lr bindings

* lr bindings usage: minor fixes

* ncm/lr: Pointer placement

* ncm: placeholder accessor cleanup

* ncm: minor fixes

* ncm: refactor rights cache

* ncm: content meta database cleanup

* ncm: move content meta database impl out of interface file

* ncm: Use const ContentMetaKey &

* ncm: fix other non-const ContentMetaKey references

* ncm: content meta database cleanup

* ncm: content storage fixes for 2.0.0

* ncm: add missing end of file newlines

* ncm: implement ContentMetaReader

* ncm: client-side api

* ncm: trim trailing spaces

* ncm: FS_MAX_PATH-1 -> fs::EntryNameLengthMax

* ncm: Use PathString and Path

* fs: implement accessor wrappers for ncm

* fs: implement user fs wrappers

* fs: add MountSdCard

* ncm: move to content manager impl

* ncm: fix up main

* kvdb: use fs::

* fs: Add wrappers needed for ncm

* ncm: use fs bindings, other refactoring

* ncm: minor fixes

* fsa: fix ReadFile without size output

* fs: add substorage, rom path tool

* ncm: fix dangling fsdev usage

* fs: fix bug in Commit

* fs: fixed incorrect mode check

* fs: implement Mount(System)Data

* ncm: don't delete hos

* results: add R_SUCCEED_IF

* ams-except-ncm: use R_SUCCEED_IF

* ncm: added comments

* ncm: fix api definitions

* ncm: use R_SUCCEED_IF

* pm: think of the savings

* ncm: employ kernel strats

* ncm: Nintendo has 5 MiB of heap. Give ourselves 4 to be safe, pending analysis

* ncm: refactor IDs, split types header into many headers

* ams.mitm: use fs bindings instead of stdio

* fs: SystemData uses SystemDataId

* ncm: improve meta-db accuracy

* ncm: inline getlatestkey

* fs: improve UnsupportedOperation results

* fs: modernize mount utils

* ams: misc fixes for merge-errors

* fs: improve unsupportedoperation results

* git subrepo pull emummc

subrepo:
  subdir:   "emummc"
  merged:   "d12dd546"
upstream:
  origin:   "https://github.com/m4xw/emuMMC"
  branch:   "develop"
  commit:   "d12dd546"
git-subrepo:
  version:  "0.4.1"
  origin:   "???"
  commit:   "???"

* util: add boundedmap

* ncm: minor style fixes

* ncm: don't unmount if mounting fails

* lr: bug fixes

* ncm: implement ncm.for-initialize + ncm.for-safemode

* lr: ncm::ProgramId::Invalid -> ncm::InvalidProgramId

* ncm: fix open directory mode on 1.0.0

* ncm: fix fs use, implement more of < 4.0.0 for-initialize/safemode

* ncm: implement packagedcontent -> content for building metadb

* ncm: fix save data flag management

* ncm: address some review suggestions (thanks @leoetlino!)

* updater: use fs bindings

* fs: implement MountCode

* fs: prefer make_unique to operator new

* ncm: implement remaining ContentMetaDatabaseBuilder functionality

Co-authored-by: Michael Scire <SciresM@gmail.com>
2020-03-08 00:06:23 -08:00

273 lines
12 KiB
C++

/*
* Copyright (c) 2018-2020 Atmosphère-NX
*
* This program is free software; you can redistribute it and/or modify it
* under the terms and conditions of the GNU General Public License,
* version 2, as published by the Free Software Foundation.
*
* This program is distributed in the hope it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
* more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include <stratosphere.hpp>
namespace ams::fssystem {
namespace {
constexpr size_t IdealWorkBufferSize = 0x100000; /* 1 MiB */
constexpr const char CommittedDirectoryPath[] = "/0/";
constexpr const char WorkingDirectoryPath[] = "/1/";
constexpr const char SynchronizingDirectoryPath[] = "/_/";
class DirectorySaveDataFile : public fs::fsa::IFile {
private:
std::unique_ptr<fs::fsa::IFile> base_file;
DirectorySaveDataFileSystem *parent_fs;
fs::OpenMode open_mode;
public:
DirectorySaveDataFile(std::unique_ptr<fs::fsa::IFile> f, DirectorySaveDataFileSystem *p, fs::OpenMode m) : base_file(std::move(f)), parent_fs(p), open_mode(m) {
/* ... */
}
virtual ~DirectorySaveDataFile() {
/* Observe closing of writable file. */
if (this->open_mode & fs::OpenMode_Write) {
this->parent_fs->OnWritableFileClose();
}
}
public:
virtual Result ReadImpl(size_t *out, s64 offset, void *buffer, size_t size, const fs::ReadOption &option) override {
return this->base_file->Read(out, offset, buffer, size, option);
}
virtual Result GetSizeImpl(s64 *out) override {
return this->base_file->GetSize(out);
}
virtual Result FlushImpl() override {
return this->base_file->Flush();
}
virtual Result WriteImpl(s64 offset, const void *buffer, size_t size, const fs::WriteOption &option) override {
return this->base_file->Write(offset, buffer, size, option);
}
virtual Result SetSizeImpl(s64 size) override {
return this->base_file->SetSize(size);
}
virtual Result OperateRangeImpl(void *dst, size_t dst_size, fs::OperationId op_id, s64 offset, s64 size, const void *src, size_t src_size) override {
return this->base_file->OperateRange(dst, dst_size, op_id, offset, size, src, src_size);
}
public:
virtual sf::cmif::DomainObjectId GetDomainObjectId() const override {
return this->base_file->GetDomainObjectId();
}
};
}
DirectorySaveDataFileSystem::DirectorySaveDataFileSystem(std::shared_ptr<fs::fsa::IFileSystem> fs)
: PathResolutionFileSystem(fs), open_writable_files(0)
{
/* ... */
}
DirectorySaveDataFileSystem::DirectorySaveDataFileSystem(std::unique_ptr<fs::fsa::IFileSystem> fs)
: PathResolutionFileSystem(std::move(fs)), open_writable_files(0)
{
/* ... */
}
DirectorySaveDataFileSystem::~DirectorySaveDataFileSystem() {
/* ... */
}
Result DirectorySaveDataFileSystem::Initialize() {
/* Nintendo does not acquire the lock here, but I think we probably should. */
std::scoped_lock lk(this->accessor_mutex);
fs::DirectoryEntryType type;
/* Check that the working directory exists. */
R_TRY_CATCH(this->base_fs->GetEntryType(&type, WorkingDirectoryPath)) {
/* If path isn't found, create working directory and committed directory. */
R_CATCH(fs::ResultPathNotFound) {
R_TRY(this->base_fs->CreateDirectory(WorkingDirectoryPath));
R_TRY(this->base_fs->CreateDirectory(CommittedDirectoryPath));
}
} R_END_TRY_CATCH;
/* Now check for the committed directory. */
R_TRY_CATCH(this->base_fs->GetEntryType(&type, CommittedDirectoryPath)) {
/* Committed doesn't exist, so synchronize and rename. */
R_CATCH(fs::ResultPathNotFound) {
R_TRY(this->SynchronizeDirectory(SynchronizingDirectoryPath, WorkingDirectoryPath));
R_TRY(this->base_fs->RenameDirectory(SynchronizingDirectoryPath, CommittedDirectoryPath));
return ResultSuccess();
}
} R_END_TRY_CATCH;
/* The committed directory exists, so synchronize it to the working directory. */
return this->SynchronizeDirectory(WorkingDirectoryPath, CommittedDirectoryPath);
}
Result DirectorySaveDataFileSystem::AllocateWorkBuffer(std::unique_ptr<u8[]> *out, size_t *out_size, size_t size) {
/* Repeatedly try to allocate until success. */
while (size > 0x200) {
/* Allocate the buffer. */
if (auto mem = new (std::nothrow) u8[size]; mem != nullptr) {
out->reset(mem);
*out_size = size;
return ResultSuccess();
} else {
/* Divide size by two. */
size >>= 1;
}
}
/* TODO: Return a result here? Nintendo does not, but they have other allocation failed results. */
/* Consider returning ResultFsAllocationFailureInDirectorySaveDataFileSystem? */
AMS_ABORT_UNLESS(false);
}
Result DirectorySaveDataFileSystem::SynchronizeDirectory(const char *dst, const char *src) {
/* Delete destination dir and recreate it. */
R_TRY_CATCH(this->base_fs->DeleteDirectoryRecursively(dst)) {
R_CATCH(fs::ResultPathNotFound) { /* Nintendo returns error unconditionally, but I think that's a bug in their code. */}
} R_END_TRY_CATCH;
R_TRY(this->base_fs->CreateDirectory(dst));
/* Get a work buffer to work with. */
std::unique_ptr<u8[]> work_buf;
size_t work_buf_size;
R_TRY(this->AllocateWorkBuffer(&work_buf, &work_buf_size, IdealWorkBufferSize));
/* Copy the directory recursively. */
return fssystem::CopyDirectoryRecursively(this->base_fs, dst, src, work_buf.get(), work_buf_size);
}
Result DirectorySaveDataFileSystem::ResolveFullPath(char *out, size_t out_size, const char *relative_path) {
R_UNLESS(strnlen(relative_path, fs::EntryNameLengthMax + 1) < fs::EntryNameLengthMax + 1, fs::ResultTooLongPath());
R_UNLESS(PathTool::IsSeparator(relative_path[0]), fs::ResultInvalidPath());
/* Copy working directory path. */
std::strncpy(out, WorkingDirectoryPath, out_size);
out[out_size - 1] = StringTraits::NullTerminator;
/* Normalize it. */
constexpr size_t WorkingDirectoryPathLength = sizeof(WorkingDirectoryPath) - 1;
size_t normalized_length;
return PathTool::Normalize(out + WorkingDirectoryPathLength - 1, &normalized_length, relative_path, out_size - (WorkingDirectoryPathLength - 1));
}
void DirectorySaveDataFileSystem::OnWritableFileClose() {
std::scoped_lock lk(this->accessor_mutex);
this->open_writable_files--;
/* Nintendo does not check this, but I think it's sensible to do so. */
AMS_ABORT_UNLESS(this->open_writable_files >= 0);
}
Result DirectorySaveDataFileSystem::CopySaveFromFileSystem(fs::fsa::IFileSystem *save_fs) {
/* If the input save is null, there's nothing to copy. */
R_SUCCEED_IF(save_fs == nullptr);
/* Get a work buffer to work with. */
std::unique_ptr<u8[]> work_buf;
size_t work_buf_size;
R_TRY(this->AllocateWorkBuffer(&work_buf, &work_buf_size, IdealWorkBufferSize));
/* Copy the directory recursively. */
R_TRY(fssystem::CopyDirectoryRecursively(this->base_fs, save_fs, PathTool::RootPath, PathTool::RootPath, work_buf.get(), work_buf_size));
return this->Commit();
}
/* Overridden from IPathResolutionFileSystem */
Result DirectorySaveDataFileSystem::OpenFileImpl(std::unique_ptr<fs::fsa::IFile> *out_file, const char *path, fs::OpenMode mode) {
char full_path[fs::EntryNameLengthMax + 1];
R_TRY(this->ResolveFullPath(full_path, sizeof(full_path), path));
std::scoped_lock lk(this->accessor_mutex);
std::unique_ptr<fs::fsa::IFile> base_file;
R_TRY(this->base_fs->OpenFile(&base_file, full_path, mode));
std::unique_ptr<DirectorySaveDataFile> file(new (std::nothrow) DirectorySaveDataFile(std::move(base_file), this, mode));
R_UNLESS(file != nullptr, fs::ResultAllocationFailureInDirectorySaveDataFileSystem());
if (mode & fs::OpenMode_Write) {
this->open_writable_files++;
}
*out_file = std::move(file);
return ResultSuccess();
}
Result DirectorySaveDataFileSystem::CommitImpl() {
/* Here, Nintendo does the following (with retries): */
/* - Rename Committed -> Synchronizing. */
/* - Synchronize Working -> Synchronizing (deleting Synchronizing). */
/* - Rename Synchronizing -> Committed. */
std::scoped_lock lk(this->accessor_mutex);
R_UNLESS(this->open_writable_files == 0, fs::ResultPreconditionViolation());
const auto RenameCommitedDir = [&]() { return this->base_fs->RenameDirectory(CommittedDirectoryPath, SynchronizingDirectoryPath); };
const auto SynchronizeWorkingDir = [&]() { return this->SynchronizeDirectory(SynchronizingDirectoryPath, WorkingDirectoryPath); };
const auto RenameSynchronizingDir = [&]() { return this->base_fs->RenameDirectory(SynchronizingDirectoryPath, CommittedDirectoryPath); };
/* Rename Committed -> Synchronizing. */
R_TRY(fssystem::RetryFinitelyForTargetLocked(std::move(RenameCommitedDir)));
/* - Synchronize Working -> Synchronizing (deleting Synchronizing). */
R_TRY(fssystem::RetryFinitelyForTargetLocked(std::move(SynchronizeWorkingDir)));
/* - Rename Synchronizing -> Committed. */
R_TRY(fssystem::RetryFinitelyForTargetLocked(std::move(RenameSynchronizingDir)));
/* TODO: Should I call this->base_fs->Commit()? Nintendo does not. */
return ResultSuccess();
}
/* Overridden from IPathResolutionFileSystem but not commands. */
Result DirectorySaveDataFileSystem::CommitProvisionallyImpl(s64 counter) {
/* Nintendo does nothing here. */
return ResultSuccess();
}
Result DirectorySaveDataFileSystem::RollbackImpl() {
/* Initialize overwrites the working directory with the committed directory. */
return this->Initialize();
}
/* Explicitly overridden to be not implemented. */
Result DirectorySaveDataFileSystem::GetFreeSpaceSizeImpl(s64 *out, const char *path) {
return fs::ResultNotImplemented();
}
Result DirectorySaveDataFileSystem::GetTotalSpaceSizeImpl(s64 *out, const char *path) {
return fs::ResultNotImplemented();
}
Result DirectorySaveDataFileSystem::GetFileTimeStampRawImpl(fs::FileTimeStampRaw *out, const char *path) {
return fs::ResultNotImplemented();
}
Result DirectorySaveDataFileSystem::QueryEntryImpl(char *dst, size_t dst_size, const char *src, size_t src_size, fs::fsa::QueryId query, const char *path) {
return fs::ResultNotImplemented();
}
Result DirectorySaveDataFileSystem::FlushImpl() {
return fs::ResultNotImplemented();
}
}