Commit graph

574 commits

Author SHA1 Message Date
Michael Scire
4b7b33809f kern: optimize and bring into line with N our pstate.i management 2021-10-27 15:00:07 -07:00
Michael Scire
e81a1ce5a8 kern: audit (and fix) our hardware maintenance instructions to match official kernel 2021-10-27 12:31:53 -07:00
Michael Scire
9cc6be4d57 kern: other dmbs in kernel were already dmb ish 2021-10-25 17:38:50 -07:00
Michael Scire
273f4a87ae kern: add (and use) generic KSystemControlBase 2021-10-25 17:31:14 -07:00
Michael Scire
2a842791eb kern: add toggleable support for 40-bit physaddr caps 2021-10-25 17:31:14 -07:00
Michael Scire
0a58e803be kern/test: add wip qemu-virt board support to mesosphere 2021-10-25 17:31:14 -07:00
Michael Scire
10ed579c38 kernel_ldr: bring initial cache flush in line with Nintendo 2021-10-25 13:38:52 -07:00
Michael Scire
6ad3219656 kern: reflect nintendo cache management behavior for initial processes 2021-10-25 13:02:35 -07:00
Michael Scire
54dde406bc kern: devirtualize KReadableEvent::Reset, KWorkerTask::DoWorkerTask 2021-10-24 20:41:38 -07:00
Michael Scire
d0cd511c0e kern: devirtualize most things that are free to devirtualize (see #1672) 2021-10-24 13:04:31 -07:00
Michael Scire
89926f44c6 kern: use optimized red black tree finds for remaining holdouts 2021-10-23 21:58:48 -07:00
Michael Scire
436613401a kern: devirtualize several KAutoObject functions 2021-10-23 21:13:26 -07:00
Michael Scire
2490bbf4f9 kern: KCacheHelper: better reflect nintendo coremask clearing logic 2021-10-23 17:44:30 -07:00
Michael Scire
71e4313d0c kern: devirtualize remaining vcall for class token/dyncast 2021-10-23 16:04:04 -07:00
SciresM
36e4914be8
kern: avoid constexpr init for many objects (avoids unnecessary memory clear) (#1668) 2021-10-23 15:25:20 -07:00
Michael Scire
20716cb3de kern: use new AtomicRef, use Atomic<bool> 2021-10-20 13:29:38 -07:00
Michael Scire
aed9d3f535 util: better match true std::atomic semantics 2021-10-20 11:02:17 -07:00
Michael Scire
d74f364107 kern/util: use custom atomics wrapper to substantially improve codegen 2021-10-19 15:24:15 -07:00
Michael Scire
52332e8d75 kern: use 13.0.0 revised KLightLock logic 2021-10-19 02:45:29 -07:00
Michael Scire
3fc695aff8 kern: fix capability check for thread priorities 2021-10-19 01:41:23 -07:00
Michael Scire
f3b532070b kern: update scheduler for 13.0.0 change, fix some ctz/clz bugs 2021-10-19 01:20:28 -07:00
Michael Scire
42b6c2dd95 kern: fix use of plr vs plr_heap, fix close/unlock order in ArbitrateLock 2021-10-19 01:19:31 -07:00
Michael Scire
889d843718 ams: improve offsetof style consistency 2021-10-18 00:17:13 -07:00
Michael Scire
bfffe6b119 kern: devirtualize KAutoObject::DynamicCast<>()
This is an optimization that saves the most common type of virtual call in the kernel (DynamicCast)
by storing class token as a member, rather than getting it via virtual call every time.

This does not currently cost any memory space on 64-bit targets, due to pre-existing padding space.

This optimization can be turned off via a compile-time flag for accuracy.
2021-10-16 16:24:06 -07:00
Michael Scire
26c02e2019 kern/util: update structure layouts to match Nintendo (saves 0x10 per KThread/KSession) 2021-10-16 16:13:10 -07:00
Michael Scire
179d91a563 kern: fix some lingering non-m_ member variables 2021-10-09 15:46:04 -07:00
Michael Scire
1ab0bd1765 ams: replace most remaining operator & with std::addressof 2021-10-09 14:49:53 -07:00
Michael Scire
ec65c39d17 strat: refactor address taking of form &var[...] 2021-10-09 12:45:31 -07:00
Michael Scire
4866e80769 svc: refactor/rename MemoryInfo fields 2021-10-05 15:16:54 -07:00
Michael Scire
719ead824e strat: remove map namespace, svc: add address space defs 2021-10-05 12:22:34 -07:00
Michael Scire
77fe5cf6f5 ams: the copyright^H^H^H^H^H^H^Hmplex plane is the algebraic closure of the reals 2021-10-04 12:59:10 -07:00
Michael Scire
3fe072a1d0 kern: devirtualize page table operations 2021-09-21 10:09:27 -07:00
Michael Scire
90732ff311 kern: unify all waiting semantics to use single api 2021-09-19 10:34:31 -07:00
Michael Scire
f6fb5f2c8d kern/svc: implement IoPool/Region svc support 2021-09-19 10:34:31 -07:00
Michael Scire
ce7dd55257 svc/kern/dd: remove MapDeviceAddressSpace() 2021-09-19 10:34:31 -07:00
Michael Scire
481ce12b7b kern: update Initialize0 for new arguments/randomization semantics 2021-09-19 10:34:31 -07:00
Michael Scire
2f2c36b22b kern: KMemoryManager/KPageGroup use physical addresses instead of virtual, now 2021-09-19 10:34:31 -07:00
Michael Scire
2c4bd44d7e kern: support dynamic resource expansion for system heaps/events/sessions. 2021-09-19 10:34:31 -07:00
Michael Scire
2b91956051 kern: improve kdebug attach semantics 2021-09-19 10:34:31 -07:00
Michael Scire
4c73c461f1 kern: update KPageTable::Unmap block closing logic 2021-09-19 10:34:31 -07:00
Michael Scire
8b49cea4a9 kern: optimize logging for release kernel strings (saves printf space in .text) 2021-09-19 10:34:31 -07:00
Michael Scire
fdf008108c kern: add new KMemoryState 2021-09-19 10:34:31 -07:00
Michael Scire
252486913b kern: KWorkerTaskManager no longer tracks id 2021-09-19 10:34:31 -07:00
Michael Scire
44d10da7b8 kern: KSchedulerInterruptTask -> KSchedulerInterruptHandler 2021-09-19 10:34:31 -07:00
Michael Scire
cb28150912 kern: kill the interrupt task manager thread 2021-09-19 10:34:31 -07:00
Michael Scire
29cc3d1c09 kern: remove per-KInterruptEventTask locks 2021-09-19 10:34:31 -07:00
Michael Scire
e6a6fe6f38 kern: delete KWritableEvent, devirtualize KReadableEvent Signal/Clear 2021-09-19 10:34:31 -07:00
Michael Scire
d80ad222cc kern: KConditionVariable arbiter functions now static 2021-09-19 10:34:31 -07:00
Michael Scire
183243bf16 kern: optimize handle table layout 2021-09-19 10:34:31 -07:00
Michael Scire
6407786059 kern: update GetInfo logic for tick count InfoTypes 2021-09-19 10:34:31 -07:00
Michael Scire
6cbfaaf835 kern: port limit is now 0x180 2021-09-19 10:34:31 -07:00
Michael Scire
ebb0bd2b41 kern: improve single-step around user-exception entry 2021-09-11 19:41:47 -07:00
Michael Scire
c10265676f kern: fix spsr register in RestoreContext 2021-09-11 19:41:47 -07:00
Michael Scire
9e7b56b33c kern: optimize hw-single-step management 2021-09-11 19:41:47 -07:00
Michael Scire
4075d24e0c kern: add hardware single step extension 2021-09-11 19:41:47 -07:00
Michael Scire
f175802136 kern: improve some debug output 2021-08-20 00:06:43 -07:00
Michael Scire
fe79bc253a kern: fix same is_current check in GetThreadContext 2021-08-19 07:38:25 -07:00
Michael Scire
81bf8c577a kern: fix check for thread-is-current in KDebugBase::SetThreadContext 2021-08-19 07:34:08 -07:00
Michael Scire
f5704d25f8 kern: remove firmare-specific SignalAndModify impl
Nintendo made this breaking change because there was zero official software relying on it,
so it's safe for us to make the same change.
2021-07-22 15:56:45 -07:00
Michael Scire
2da31b122f kern: fix hardware watchpoint detection 2021-07-22 00:35:40 -07:00
Michael Scire
43bbfd29bb kern: fix inverted condition in context breakpoint validation 2021-07-21 23:56:28 -07:00
Michael Scire
0c596e682f exo/daybreak: advertise (and check against) supported hos version 2021-07-21 18:21:38 -07:00
Michael Scire
bd6155bcb4 kern: since 10.0.0, KDebug::GetThreadContext always returns X0-X7 2021-07-13 13:00:16 -07:00
Michael Scire
3bedf56512 kern: fix incorrect waiter management in KProcess::ReleaseUserException 2021-07-13 05:27:19 -07:00
Michael Scire
1be74ea6e2 kern: fix bug in KConditionVariable::SignalImpl 2021-07-13 01:40:16 -07:00
Michael Scire
14d458522d kern: update initial cache management to match latest kernel 2021-07-12 18:30:01 -07:00
Michael Scire
5e3339e866 kern: add missing abort on unmap failure 2021-07-11 21:59:06 -07:00
Michael Scire
2088c5a34a kern: fix handling of is_aligned parameter in DevicePageTable 2021-07-04 10:45:14 -07:00
Michael Scire
4892ffae15 kern: implement improved [new page tables are zero] invariant 2021-06-17 13:03:46 -07:00
Michael Scire
25305257d6 kern: fix missing peak set in KResourceLimit::SetLimitValue 2021-06-17 12:57:56 -07:00
Michael Scire
7821241356 kern: fix enormous whoops 2021-05-31 04:20:59 -07:00
Michael Scire
dcdf46f576 kern: slightly improve genericity of debugger break event 2021-05-30 21:13:42 -07:00
Michael Scire
4f16106702 exo/meso: update for gcc 11 compatibility 2021-04-26 20:06:18 -07:00
Michael Scire
19be54ff95 kern: fix initial process binary load on 2.0.0-4.1.0 (closes #1460) 2021-04-21 19:24:41 -07:00
Michael Scire
57b6c71c1c util: implement red black trees as templates over macros 2021-04-20 16:56:33 -07:00
Michael Scire
fbc526d163 kern: tweak KAutoObject::Open/Close codegen 2021-04-19 18:03:27 -07:00
Michael Scire
037b04ac60 kern: mostly kill magic numbers in assembly, fix SVCs >= 0x80 2021-04-14 18:01:08 -07:00
Michael Scire
96937a611d kern: fuck the KPolice^H^H^H^H^H^HPageGroups 2021-04-11 03:42:16 -07:00
Michael Scire
6faa3534bf kern: update pinning semantics for terminating threads 2021-04-11 03:42:16 -07:00
Michael Scire
afb1d68d06 kern: ensure handle table is finalized when deferring termination 2021-04-11 03:42:16 -07:00
Michael Scire
911e431d65 kern: simplify handle table registration for port/session 2021-04-11 03:42:16 -07:00
Michael Scire
ee91063bbb kern: update kdebug process management semantics 2021-04-11 03:42:16 -07:00
Michael Scire
cbdf33260e kern: update port/session state semantics 2021-04-11 03:42:16 -07:00
Michael Scire
c62a7381f8 kern: update KLightConditionVariable 2021-04-11 03:42:16 -07:00
Michael Scire
b4498734e4 kern: optimize KHandleTable to use indices instead of pointers 2021-04-11 03:42:16 -07:00
Michael Scire
4407237f5b kern: KAutoObject destruction is now scheduled for next dpc-time 2021-04-11 03:42:16 -07:00
Michael Scire
15956fcf9a kern: update for new slab resource counts/extents 2021-04-11 03:42:16 -07:00
Michael Scire
6a368d3d1a kern: reallocate pool distributions for 8GB units 2021-04-11 03:42:16 -07:00
Michael Scire
8e4be9aef9 kern: simplify global rng initialization 2021-04-11 03:42:16 -07:00
Michael Scire
0f8b7be2d2 kern: load initial process binary from user pool, rather than from pt heap 2021-04-11 03:42:16 -07:00
Michael Scire
a1e137cc1c kern: update Initialize0 for new changes 2021-04-11 03:42:16 -07:00
Michael Scire
504472af4e kern: update KConditionVariable::WaitForAddress/Wait 2021-04-11 03:42:16 -07:00
Michael Scire
19b253fd17 kern: trivially optimize userspace io memory write 2021-04-11 03:42:16 -07:00
Michael Scire
01f5c89902 kern: add bounds checking to KHandleTable::Register/Unreserve 2021-04-11 03:42:16 -07:00
Michael Scire
44ccbc2a7b kern: update set/way cache operations for new semantics 2021-04-11 03:42:16 -07:00
Michael Scire
6e4664ee05 kern: if a page table region is zero-size, nothing overlaps it 2021-04-11 03:42:16 -07:00
Michael Scire
85f9355184 kern: use KScopedLightLockPair helper for page table pair-locks 2021-04-11 03:42:16 -07:00
Michael Scire
60b5bd73b7 kern: track mapped ipc server memory in page table 2021-04-11 03:42:16 -07:00
Michael Scire
53e7aa0a20 kern: add KPageTableBase::Read/WriteDebugIoMemory 2021-04-11 03:42:16 -07:00
Michael Scire
561a16a348 kern: flush memory before reading in KPageTableBase::ReadDebugMemory 2021-04-11 03:42:16 -07:00
Michael Scire
1fce7b08b1 kern: update KMemoryBlockManagerUpdaterAllocator for new ctor/init semantics 2021-04-11 03:42:16 -07:00
Michael Scire
c216f92a91 kern: swap tpidr_el1/cntv_cval_el0 as scratch vs exception stack 2021-04-11 03:42:16 -07:00
Michael Scire
2f930c2d5f kern: support immortal processes 2021-04-11 03:42:16 -07:00
Michael Scire
256eb92f4c kern: update process/thread for new running/termination semantics 2021-04-11 03:42:16 -07:00
Michael Scire
ec1d9c4c49 kern: unconditionally set thread state when appropriate 2021-04-11 03:42:16 -07:00
Michael Scire
3356eddcba kern: update kernel waiter management rules 2021-04-11 03:42:16 -07:00
Michael Scire
f67d1b7026 kern: update KInterruptEvent to store core id 2021-04-11 03:42:16 -07:00
Michael Scire
e64fef109c kern: update pinned thread priority rules 2021-04-11 03:42:16 -07:00
Michael Scire
1b2cf173b3 kern: add new checks to SetThreadPriority/CoreMask 2021-04-11 03:42:16 -07:00
Michael Scire
962cf97150 kern: KLinkedList no longer exists 2021-04-11 03:42:16 -07:00
Michael Scire
4b9e7c7d27 kern: bump svc limit to 192 from 128 2021-04-11 03:42:16 -07:00
Michael Scire
d84dcb653d ams: prefer construct_at/destroy_at over placement new/explicit destructor 2021-03-21 20:30:40 -07:00
Michael Scire
a7564cf303 kern: add extension InfoType for retrieving current process handle. 2021-03-17 17:48:30 -07:00
Michael Scire
021d4c88fa kern: use fix usage of incorrect page table for UserBuffer ipc 2021-03-13 15:14:36 -08:00
Michael Scire
deb4aece9a kern: fix inverted conditional in KDebugBase::SetThreadContext 2021-03-11 12:53:43 -08:00
Michael Scire
8b32b9eadf kern: Increase reserved system memory, require mesosphere for htc/tma 2021-02-24 04:06:54 -08:00
Michael Scire
eb50e99748 kern: alleviate a little KPort pressure. 2021-02-23 14:31:24 -08:00
Michael Scire
621520c30b kern: fix support for virtual core IDs 2021-02-05 14:59:03 -08:00
Michael Scire
4ce2a6deb3 meso: update for new fatal encoding 2021-02-02 11:41:49 -08:00
Michael Scire
a28c60970f kern: fix scheduler update semantics in KLightLock::LockSlowPath 2021-01-28 17:47:26 -08:00
Michael Scire
1e643f7ab0 kern: further codegen tweaks 2021-01-08 02:35:29 -08:00
Michael Scire
4aa18b06e8 kern: greatly improve codegen for atomics, scheduler 2021-01-08 02:13:43 -08:00
Michael Scire
1c9d6b4d90 kern: fix svc bounds checking for main memory size 4GB -> 8GB (closes #1320) 2021-01-07 03:43:09 -08:00
Michael Scire
113ab8439d kern: fix KTrace buffer memory region type to match official kernel 2020-12-28 00:57:45 -08:00
ameerj
60c888126b fix typo in KServerPort::Destroy() 2020-12-25 20:23:10 -08:00
Michael Scire
968f50bc07 kern: refactor to use m_ for member variables 2020-12-18 13:31:01 -08:00
Michael Scire
4ff026224e kern: fix inverted conditional in waiter debug print 2020-12-14 10:32:49 -08:00
Michael Scire
16cd1141d1 kern: fix some typos/naming semantics 2020-12-12 05:36:48 -08:00
Michael Scire
4b71e3a87e kern: implement process cpu utilization debug 2020-12-12 05:34:31 -08:00
Michael Scire
d090a7886d kern: add kernel thread utilization debug 2020-12-12 05:02:23 -08:00
Michael Scire
9a6cca7499 kern: add page table contents debug 2020-12-11 19:30:48 -08:00
Michael Scire
569e1bc650 kern: cfi fix pt 2 2020-12-11 04:59:20 -08:00
Michael Scire
652e9bf9e8 kern: fix cfi (requires x18 not cleared on return to userland) 2020-12-11 04:53:25 -08:00
Michael Scire
17c8c390fc kern: fix building debug config 2020-12-11 03:48:34 -08:00
Michael Scire
7fb902d8fb kern: update for 11.0.1 2020-12-11 02:59:09 -08:00
Michael Scire
be8473cf65 kern: implement memory debug 2020-12-10 16:32:19 -08:00
Michael Scire
6df26d674c kern/sm: fix debug port output 2020-12-10 04:06:02 -08:00
Michael Scire
1a6e003a5d kern: add kernel object debug 2020-12-10 03:31:57 -08:00
Michael Scire
0acd79c8c2 kern: implement port debug 2020-12-10 01:44:27 -08:00
Michael Scire
8a4bf6a0a8 kern: add handle table/process/suspend/resume debug 2020-12-09 23:44:36 -08:00
Michael Scire
af259eabda kern: implement thread call stack debug 2020-12-09 22:49:04 -08:00
Michael Scire
bcc7eed037 kern: add debug thread dump 2020-12-09 05:59:54 -08:00
Michael Scire
abd7ad2720 meso: properly initialize per-thread CFI-value for 11.x 2020-12-08 16:16:49 -08:00
Michael Scire
2de85c633a exo/meso/fusee: support dynamic control of log port/baud rate 2020-12-07 19:25:06 -08:00
Michael Scire
c45088d1cd kern: add support for InfoType_FreeThreadCount 2020-12-04 18:20:56 -08:00
Michael Scire
bba99d49da kern: fix inverted conditional in UnmapProcessCodeMemory 2020-12-02 11:31:50 -08:00
Michael Scire
e760a9d4b0 kern: fix reboot to fatal error 2020-12-02 11:31:15 -08:00
Michael Scire
726d7b6e4d kern: tweak optimization settings for hot paths 2020-12-02 06:43:37 -08:00
Michael Scire
ab8de72db0 microkernel: hot paths are pretty fucking hot 2020-12-02 06:43:37 -08:00
Michael Scire
331fa1d00d kern: update KConditionVariable to support new has_waiter_flag rules 2020-12-02 06:43:37 -08:00